In a recent study published in JAMA Network Open, researchers investigated whether large language models (LLMs) might improve the understandability and readability of hospital discharge summaries.
Study: Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and Format. Image Credit: Chinnapong/Shutterstock.com
Background
The 21st Century Cures Act requires patients to have access to their clinical notes and other information in electronic health records. LLMs based on generative artificial intelligence (AI) may convert medical discharge notes into a more user-friendly format, increasing patient participation and trust in their treatment.
According to several studies, better readability in discharge summaries may improve outcomes such as readmissions. AI-generated plain English notes may be more usable, beneficial to the patient-clinician bond, and empower individuals, allowing for high-quality care.
However, incorporating generative AI technology into medical care raises patient safety and physician liability problems, with mistakes in AI content being a primary source of concern.
About the study
In the present study, researchers investigated using generative artificial intelligence technologies to convert medical reports into patient-friendly notes.
The researchers examined discharge records for 50 individuals receiving discharge from New York University’s General Internal Medicine unit between June 1 and 30, 2023, omitting those discharged as dead. The sample did not contain more than one discharge from a single subject.
The research intervention was a Health Insurance Portability and Accountability Act-adapted platform designed to make discharge summaries more patient-friendly.
The team evaluated readability using Flesch-Kincaid grades and comprehensibility using the Patient Education Materials Assessment Tool (PEMAT). They also assessed the completeness and correctness of the modified patient-friendly versions of discharge reports.
A single reviewer validated the authenticity of the discharge summaries created by the General Internal Medicine Service.
Discharge summaries, as they exist in the medical record, known as original discharge reports, were the inputs processed by the generative AI program to generate patient-friendly notes as the output.
Between July 26 and August 5, 2023, the LLM processed and transformed the hospital discharge reports into a patient-friendly format, which they then compared.
The patient-friendly summaries contained elements such as hospitalization and discharge dates, indications for admission, history of the presenting disease, hospital course, diagnosis, procedures, and discharge physician.
The admitting physician, hospitalization source, coding and billing tables, diagnostic assessments, and discharge status were all omitted from the converted summaries.
The converted patient-friendly versions were one-page, question-and-answer style. From May 24 to July 13, 2023, physician data scientists, informaticists, internal medicine experts, resident physicians, and AI developers worked together to create the AI prompt.
The team collaborated to generate a discharge report for an acute care medicine unit to stay accessible to a sixth-grade reading-level patient. Two resident doctors independently examined each discharge report.
Results
The researchers included the discharge reports of 50 individuals [median age, 66 years; 19 males (38%) and 31 females (62%)].
Ethnic and racial identification in electronic medical records of the patients included 50% (n=25) white individuals, 30% (n=15) African American or black individuals, 4.0% (n=2) Asians, and 16% (n=8) individuals from other races or ethnicities.
The AI-converted patient-friendly notes had significantly fewer words than the discharge reports (mean: 338 versus 1,520 words). Readability judged by Flesch-Kincaid Reading Ease scores was much better in patient-friendly notes than in original discharge reports (mean: 70 versus 36).
Similarly, PEMAT understandability ratings were much higher in patient-friendly notes (81% versus 13%). The average Flesch-Kincaid grades were much lower in the patient-friendly versions of the discharge reports (6.2 versus 11).
Two physicians evaluated the transformed patient-friendly notes for correctness, with 54 out of 100 evaluations (54%) providing the highest possible grade of 6.0, with 48% inter-rater reliability for top box precision.
In 56 discharge note reviews, the researchers graded the summaries as complete (56%). Procedures (40%) and history of current disease (25%) were the most commonly assessed as incomplete categories, with interrater reliability of 88% across categories.
Eighteen reviews (39%) raised safety issues; most of them included omissions (52%), although there were also some false claims or hallucinations (8.7%).
Based on the study findings, generative AI-based LLMs may convert discharge reports into a patient-friendly form, making them more accessible and intelligible.
However, the models need higher accuracy, completeness, and safety. The safety concerns warrant physician review throughout the initial deployment phase. The large word count of original discharge summaries may make them inaccessible, making balancing accuracy and thoroughness critical.
Future versions will need to investigate the trade-off between readability and thoroughness, as well as non-English outputs and human-generated, patient-friendly notes of discharge reports.
Study shows community health workers can improve asthma care for children