Readability Formulas and User Perceptions of Electronic Health Records Difficulty: A Corpus Study

Abstract

BackgroundElectronic health records (EHRs) are a rich resource for developing applications to engage patients and foster patient activation, thus holding a strong potential to enhance patient-centered care. Studies have shown that providing patients with access to their own EHR notes may improve the understanding of their own clinical conditions and treatments, leading to improved health care outcomes. However, the highly technical language in EHR notes impedes patients’ comprehension. Numerous studies have evaluated the difficulty of health-related text using readability formulas such as Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), and Gunning-Fog Index (GFI). They conclude that the materials are often written at a grade level higher than common recommendations.ObjectiveThe objective of our study was to explore the relationship between the aforementioned readability formulas and the laypeople’s perceived difficulty on 2 genres of text: general health information and EHR notes. We also validated the formulas’ appropriateness and generalizability on predicting difficulty levels of highly complex technical documents.MethodsWe collected 140 Wikipedia articles on diabetes and 242 EHR notes with diabetes International Classification of Diseases, Ninth Revision code. We recruited 15 Amazon Mechanical Turk (AMT) users to rate difficulty levels of the documents. Correlations between laypeople’s perceived difficulty levels and readability formula scores were measured, and their difference was tested. We also compared word usage and the impact of medical concepts of the 2 genres of text.ResultsThe distributions of both readability formulas’ scores (P<.001) and laypeople’s perceptions (P=.002) on the 2 genres were different. Correlations of readability predictions and laypeople’s perceptions were weak. Furthermore, despite being graded at similar levels, documents of different genres were still perceived with different difficulty (P<.001). Word usage in the 2 related genres still differed significantly (P<.001).ConclusionsOur findings suggested that the readability formulas’ predictions did not align with perceived difficulty in either text genre. The widely used readability formulas were highly correlated with each other but did not show adequate correlation with readers’ perceived difficulty. Therefore, they were not appropriate to assess the readability of EHR notes.

Integrating Patient-Centered Electronic Health Record Communication Training into Resident Onboarding: Curriculum Development and Post-Implementation Survey Among Housestaff

“…Patient-centered care has been found to positively impact a number of measures such as patient compliance, satisfaction, and health outcomes [ 13 ]. Given the increasing rates of EHR adoption, it is critical to provide physicians with formal education on patient-centered EHR communication skills; however, finding the time to teach these skills in crowded training programs poses a challenge.…”

“…As a result, patient satisfaction with the interaction and the perception of their overall quality of care may be decreased [ 4 - 11 ]. Similarly, best practices for patient-physician EHR interactions exist, and research shows that when EHRs are integrated in a patient-centered manner, physicians can foster patient activation and engagement by improving communication, enhancing patient understanding of their medical conditions, and promoting increased shared decision making [ 1 , 4 , 12 , 13 ].…”

Designing an Extensible Domain-Specific Web Corpus for “Layfication”

“…Their results showed that this percentage correlates significantly with the level of understanding as indicated by users but not with the readability formula levels. On the same line, Zheng & Yu (2017) found that the correlations of readability predictions and laypeople's perceptions were weak. Their study with English texts explored the relationship between several readability formulas and the laypeople's perceived difficulty on two genres of text: general health information and electronic health record (EHR) notes.…”

Consumer Health Search on the Web: Study of Web Page Understandability and Its Integration in Ranking Algorithms (Preprint)

“…For example, Becker reported that the majority of health Web sites are not well designed for the elderly [21], while Stossel et al found that health education material on the Web is not written at an adequate reading level [18]. Zheng and Yu have reported on the readability of electronic health records compared to Wikipedia pages related to diabetes and found that readability measures often do not align with user ratings of readability [22]. A common finding of these studies is that, in general, health content available on Web pages is often hard to understand by the general public; this includes content that is retrieved in topranked positions by current commercial search engines [4,5,6,7,8,9].…”