GPT-4 shows promise in locating brain lesions after stroke

Artificial intelligence (AI) may serve as a future tool for neurologists to help locate where in the brain a stroke occurred. In a new study, AI processed text from health histories and neurologic examinations to locate lesions in the brain. The study, which looked specifically at the large language model called generative pre-trained transformer 4 (GPT-4), is published in the March 27, 2024, online issue of Neurology^® Clinical Practice, an official journal of the American Academy of Neurology.

A stroke can cause long-term disability or even death. Knowing where a stroke has occurred in the brain helps predict long-term effects such as problems with speech and language or a person's ability to move part of their body. It can also help determine the best treatment and a person's overall prognosis.

Damage to the brain tissue from a stroke is called a lesion. A neurologic exam can help locate lesions, when paired with a review of a person's health history. The exam involves symptom evaluation and thinking and memory tests. People with stroke often have brain scans to locate lesions.

Not everyone with stroke has access to brain scans or neurologists, so we wanted to determine whether GPT-4 could accurately locate brain lesions after stroke based on a person's health history and a neurologic exam."

Jung-Hyun Lee, MD, study author of State University of New York (SUNY) Downstate Health Sciences University in Brooklyn and member of the American Academy of Neurology

The study used 46 published cases of people who had stroke. Researchers gathered text from participants' health histories and neurologic exams. The raw text was fed into GPT-4. Researchers asked it to answer three questions: whether a participant had one or more lesions; on which side of the brain lesions were located; and in which region of the brain the lesions were found. They repeated these questions for each participant three times. Results from GPT-4 were then compared to brain scans for each participant.

Researchers found that GPT-4 processed the text from the health histories and neurologic exams to locate lesions in many participants' brains, identifying which side of the brain the lesion was on, as well as the specific brain region, with the exception of lesions in the cerebellum and spinal cord.

For the majority of people, GPT-4 was able to identify on which side of the brain lesions were found with a sensitivity of 74% and a specificity of 87%. Sensitivity is the percentage of actual positives that are correctly identified as positive. Specificity is the percentage of negatives that are correctly identified. It also identified the brain region with a sensitivity of 85% and a specificity of 94%.

When looking at how often the three tests had the same result for each participant, GPT-4 was consistent for 76% of participants regarding the number of brain lesions. It was consistent for 83% of participants for the side of the brain, and for 87% of participants regarding the brain regions.

However, when combining its responses to all three questions across all three times, GPT-4 provided accurate answers for 41% of participants.

"While not yet ready for use in the clinic, large language models such as generative pre-trained transformers have the potential not only to assist in locating lesions after stroke, they may also reduce health care disparities because they can function across different languages," said Lee. "The potential for use is encouraging, especially due to the great need for improved health care in underserved areas across multiple countries where access to neurologic care is limited."

A limitation of the study is that the accuracy of GPT-4 depends on the quality of the information it is provided. While researchers had detailed health histories and neurologic exam information for each participant, such information is not always available for everyone who has a stroke.