In a recent study posted to the medRxiv preprint server, researchers investigate the attitudes, familiarity, benefits, constraints, and factors influencing artificial intelligence chatbot (AIC) utilization in scientific research.
Study: Attitudes and Perceptions of Medical Researchers Towards the Use of Artificial Intelligence Chatbots in the Scientific Process: A Large-Scale, International Cross-Sectional Survey. Image Credit: meeboonstudio / Shutterstock.com
*Important notice: medRxiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as conclusive, guide clinical practice/health-related behavior, or treated as established information.
The pros and cons of AI in science
AI has a crucial role in scientific research by automating operations such as literature searches and paper authoring. Chatbots are AI algorithms that replicate human discussions, which can be used to improve readability, equality, and research variety. Moreover, chatbots can distinguish between reproducible and non-reproducible experiments by calculating the replication likelihood.
However, AI chatbots are associated with certain limitations in regards to their accuracy, dependability, ethical concerns such as plagiarism, research fraud, copyright, and a lack of transparency, thus leading to disinformation and negative influences.
About the study
In the present cross-sectional study, researchers investigate medical researchers' attitudes and perceptions toward AIC use in scientific research. To this end, a web-based poll was provided to published medical researchers, which led to a total of 61,560 author names and email addresses. Journals indexed in MEDLINE were identified and used to extract a total of 122,323 PubMed Identifier (PMID) numbers from all generated articles.
Medical researchers with at least one terminal degree in their field of study and over five years of experience in a research-focused profession were included in the study, with students excluded. Survey respondents were included through convenience sampling and issued email invitations on July 9, 2023, with two reminder emails in the event of no response. Study participants completed the survey within three weeks through August 11, 2023.
The survey included 29 questions on demographics, AIC experience, functions, advantages, and perceived issues concerning scientific research. Respondents were able to offer further comments and criticism on the AIC use for scientific purposes. Percentages were used to summarize quantitative data, whereas qualitative data gathered through open-ended questions were studied thematically.
Study findings
Among the 61,560 email subscribers, 2,452 replied, which led to a response rate of 4%, 95% of whom completed the survey. About 54% of the respondents were male, 33% were between 36 and 45 years of age, and 28% resided in the United States.
About 53% of the study respondents were senior researchers, 63% were faculty members at universities or academic institutions, 51% were primarily engaged in clinical research, and 72% of the study respondents had over 21 publications.
Moreover, 61% of respondents were aware of AICs, with 66% reporting ChatGPT utilization. About 52% of the respondents had previously used AIC for scientific reasons. Notably, 24% of respondents indicated that their academic institution provided training on utilizing AI technologies, with 66% of these individuals completing the program.
About 70% of the study cohort were interested in further learning and obtaining scientific AIC use training. Of these individuals, 36% and 46% believed that AICs would be highly significant or relevant, respectively, in the future of scientific research.
The respondents reported varied feelings about AIC usage benefits; however, most respondents agreed on its limitations and challenges. About 60% of the study cohort believed that AICs would have a very beneficial or positive effect, whereas 19% believed that chatbots would have a bad or very negative influence on future scientific studies.
Among the respondents, 67% agreed that AIC were most useful in decreasing researchers' administrative burden and workload. However, a lack of understanding of AIC decision-making and response generation was a significant concern for 77% of respondents.
About 60% of respondents indicated that AICs would be extremely helpful or useful for writing or editing publications, research funding applications, and translating research materials into another language. Respondents expressed mixed feelings about the utility of AICs in understanding or selecting a research approach, 40% of whom believed that AICs were not helpful.
Most of the respondents agreed that AIC benefits included reduced administrative burden and workload at 67%. Additional reported benefits associated with AIC included more inclusive research (58%), constant access to scientific data and assistance (56%), increased data handling efficiency as compared to humans (56%), enhanced speed and efficiency of analysis (53%), increased efficiency and quality of scientific deliberation (52%), and cost-effectiveness (50%).
The opinions on whether AIC use may enhance research transparency and reproducibility varied. More specifically, 29% of respondents were doubtful that AICs would improve data analysis and experimentation accuracy, whereas 27% did not believe that these chatbots would eliminate human error and bias by offering a systematic method of data analysis.
The most commonly reported AIC challenges were legal and ethical concerns, lack of understanding in decision-making and response generation, lack of interpretability and transparency in decision-making, and limited capturing the complexities and nuances of human reasoning and thought at over 76% each. Other challenges were related to user acceptance and adoption at 75%, limited dealing with context-dependent or situation-specific data at 74%, lack of accountability at 73%, data privacy concerns at 73%, biased data outputs at 72%, and researcher resistance or pushback at 71%.
Conclusions
Although most study respondents were aware of AI capabilities, many were hesitant due to its limitations. Nevertheless, ChatGPT remains the most popular AI tool, with most respondents finding it beneficial in manuscript drafting and editing.
Importantly, there is limited formal training in applying AI technologies in scientific research across academic organizations. The lack of AI training raises issues about research integrity.
*Important notice: medRxiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as conclusive, guide clinical practice/health-related behavior, or treated as established information.
Major donation fuels advances in ophthalmic artificial intelligence