Spanish-language bariatric surgery patient education materials fail to meet healthcare literacy standards of readability

Padilla, Geraldine; Awshah, Sabrina; Mhaskar, Rahul; Diab, Abdul-Rahman F.; Sujka, Joseph; DuCoin, Christopher; Docimo, Salvatore

doi:10.1007/s00464-023-10088-9

Cited by 3 publications

(1 citation statement)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our analysis shows that institutional websites’ PEMs remain too complex for the public, falling short of the AMA recommendation that PEMs be written at a 6th grade level or below [ 13 ]. These findings echo the results of previous studies that showed poor readability of bariatric surgery PEMs online [ 47 , 48 ]. Furthermore, initial responses from LLMs were also found to have poor readability and in some instances worse readability than the institutions.…”

Section: Discussionsupporting

confidence: 91%

Large language models and bariatric surgery patient education: a comparative readability analysis of GPT-3.5, GPT-4, Bard, and online institutional resources

Srinivasan,

Samaan,

Rajeev

et al. 2024

Surg Endosc

View full text Add to dashboard Cite

Background The readability of online bariatric surgery patient education materials (PEMs) often surpasses the recommended 6th grade level. Large language models (LLMs), like ChatGPT and Bard, have the potential to revolutionize PEM delivery. We aimed to evaluate the readability of PEMs produced by U.S. medical institutions compared to LLMs, as well as the ability of LLMs to simplify their responses. Methods Responses to frequently asked questions (FAQs) related to bariatric surgery were gathered from top-ranked health institutions. FAQ responses were also generated from GPT-3.5, GPT-4, and Bard. LLMs were then prompted to improve the readability of their initial responses. The readability of institutional responses, initial LLM responses, and simplified LLM responses were graded using validated readability formulas. Accuracy and comprehensiveness of initial and simplified LLM responses were also compared. Results Responses to 66 FAQs were included. All institutional and initial LLM responses had poor readability, with average reading levels ranging from 9th grade to college graduate. Simplified responses from LLMs had significantly improved readability, with reading levels ranging from 6th grade to college freshman. When comparing simplified LLM responses, GPT-4 responses demonstrated the highest readability, with reading levels ranging from 6th to 9th grade. Accuracy was similar between initial and simplified responses from all LLMs. Comprehensiveness was similar between initial and simplified responses from GPT-3.5 and GPT-4. However, 34.8% of Bard's simplified responses were graded as less comprehensive compared to initial. Conclusion Our study highlights the efficacy of LLMs in enhancing the readability of bariatric surgery PEMs. GPT-4 outperformed other models, generating simplified PEMs from 6th to 9th grade reading levels. Unlike GPT-3.5 and GPT-4, Bard’s simplified responses were graded as less comprehensive. We advocate for future studies examining the potential role of LLMs as dynamic and personalized sources of PEMs for diverse patient populations of all literacy levels.

show abstract

Section: Discussionsupporting

confidence: 91%