In a recent study published in JAMA Network Open, researchers described a comparative evaluation of four different voice assistants (VAs) for their ability to provide appropriate cardiopulmonary resuscitation (CPR) instructions.
Study: Quality of Layperson CPR Instructions From Artificial Intelligence Voice Assistants. Image Credit: Platoo Studio/Shutterstock.com
The ability of a recently developed artificial intelligence-based large language model (LLM) to provide cardiopulmonary resuscitation guidance to laypersons was also tested.
Background
Despite extensive cardiopulmonary resuscitation training, layperson cardiopulmonary resuscitation is conducted in a significant number of out-of-hospital cardiac arrest situations.
The use of layperson cardiopulmonary resuscitation increases survival. AI voice assistants are becoming more common and may be an innovative way to offer spoken cardiac resuscitation instructions during an emergency.
Although emergency dispatchers can provide cardiopulmonary resuscitation guidance to onlookers, such services may not be universally accessible, and their usage may be hampered by language hurdles, poor quality of audio, disconnection of calls, perceived expenses, and law enforcement-related fears.
In situations where alternative sources of cardiopulmonary resuscitation training are unavailable, the voice assistant may provide easily available cardiopulmonary resuscitation instructions.
About the study
In the present study, researchers reported on the use of artificial intelligence-based voice assistants and LLM models to provide layperson cardiopulmonary resuscitation guidance.
The study did not involve humans. In February of 2023, the team tested multiple ways of inquiring about cardiopulmonary resuscitation on four different voice assistants. In total, eight verbal queries were made to every voice assistant.
The researchers reviewed transcriptions of the interactions to confirm that the questions were captured accurately. In addition, the questions were asked to an AI-based LLM tool on 13 February via text.
Two skilled emergency medicine health physicians assessed response quality. The queries included the way in which CPR must be performed, the way to perform chest compressions, help with breathlessness, and ways to help an individual who does not have a pulse.
The voice assistants used were Amazon Alexa, Google Assistant, Microsoft Cortana, and Apple Siri. The ChatGPT LLM tool was used for comparative evaluation.
Results
Of 32 digital responses, 19 (59%) were associated with CPR, nine (28%) suggested seeking help from emergency medical services, four (12%) provided verbal instructions, and 11 (34%) provided textual or verbal cardiopulmonary resuscitation instructions. Differences were noted in the applicability of the responses. As an example, chatGPT showed a higher frequency of providing cardiopulmonary resuscitation instructions than its counterparts; however, the instructions were limited to text messages.
The large language model provided relevant cardiopulmonary resuscitation data for all questions and textual cardiopulmonary resuscitation instructions for three-quarters of questions. Of 17 answers from the voice assistants and the large language model, 71%, 47%, and 35% were related to hand positioning, the depth of compression, and the rate of compression, respectively.
As an example, among the VA responses, 28% suggested using emergency medical services, indicating that using voice assistants might result in delays in seeking appropriate medical care. The large language model performed better than the digital voice assistants; however, the model provided inconsistent responses.
Of eight questions asked, Amazon Alexa provided CPR-related answers, three unrelated answers, and one ‘do not know’ answer. Apple Siri provided six CPR-related and two unrelated answers. Google Assistant provided five CPR-related answers, and three ‘do not know’ answers. Microsoft Cortana provided four CPR-related, two unrelated, and two ‘do not know’ answers.
Among voice assistants, verbal CPR instructions were only provided by Amazon Alexa and Google Assistant. While Apple Siri provided most CPR-related answers, Microsoft Cortana provided the most CPR instructions. In comparison to the four different voice assistants, chatGPT provided more CPR-related answers, CPR instructions, and emergency service recommendations.
Implications
Overall, the study findings showed that almost 70% of questions were responded to by voice assistants with data unrelated to cardiopulmonary resuscitation, often with largely unsuitable responses, indicating that laypersons seeking cardiopulmonary resuscitation guidance may encounter either delay or be unable to find relevant and precise content.
The findings indicate that onlookers must prioritize contacting emergency medical services over utilizing a voice assistant, especially since bystanders might not identify a cardiac arrest patient.
Even dispatchers may find this recognition difficult, and it is possible that machine learning can help with this as well. Future research must be conducted, including more questions and an evaluation of changes in responses with time.
Voice assistants must improve their support for cardiopulmonary resuscitation by (i) incorporating cardiopulmonary resuscitation instructions in their core functions; (ii) identifying common phrases for providing cardiopulmonary resuscitation instructions; and (iii) establishing common evidence-backed content across all digital assistance devices, including the prioritization of seeking emergency medical care for suspected cases of cardiac arrest.
To standardize voice assistant support for cardiopulmonary resuscitation education, the technology sector might collaborate with professional associations and the health community.
Black caregivers express concerns over police interactions with autistic children