Assessing the Utility of ChatGPT in Simplifying Text Complexity of Patient Educational Materials

Abstract

Introduction: AI chatbots are being increasingly used in healthcare settings. There is growing interest in using AI to assist in patient education. Currently, extensive healthcare information is found online but is often too complex to understand. Our objective is to determine if physicians can recommend the free version of ChatGPT version 3.5 (OpenAI, San Francisco, CA, USA) for patients to simplify text from the American Academy of Ophthalmology (AAO) in English and Spanish. This version of ChatGPT was assessed in this study due to its increased accessibility across various patient populations. Methods: Fifteen articles were chosen from AAO in both languages and simplified with ChatGPT 10 times each. The readability of original and simplified articles was assessed with the Flesch Reading Ease and Gunning Fog Index for English and Fernández Huerta, Gutiérrez, Szigriszt-Pazo, INFLESZ, and Legibilidad-µ for Spanish. Grade levels to assess readability were calculated with Flesch Kincaid Grade Level and Crawford Nivel-de-Grado. Mean, standard deviation, and two-tailed t-tests were performed to assess differences before and after simplification. Results: Average grade levels before and after simplification were as follows: English 8.43±1.17 to 8.9±2.1 (p=0.41) and Spanish 5.3±0.34 to 4.1±1.1 (p=0.0001). Spanish articles were significantly simplified per Legibilidad-µ (p=0.003). No significant difference was noted for other scales. Conclusions: The readability of AAO articles in English worsened without significance but significantly improved in Spanish. This may result from simpler syllable structures and a lesser overall vocabulary in Spanish. With increased testing, physicians can recommend ChatGPT for Spanish-speaking patients to improve health literacy.