Multiple Automated Health Literacy Assessments of Written Health Information: Development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor v1

Abstract

Producing health information that people can easily understand is challenging and time-consuming. Existing guidance is often subjective and lacks specificity. With advances in software that reads and analyzes text, there is an opportunity to develop tools that provide objective, specific, and automated guidance on the complexity of health information. This paper outlines the development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor, an automated tool to facilitate the implementation of health literacy guidelines for the production of easy-to-read written health information. Target users were any person or organization that develops consumer-facing education materials, with or without prior experience with health literacy concepts. Anticipated users included health professionals, staff, and government and nongovernment agencies. To develop this tool, existing health literacy and relevant writing guidelines were collated. Items amenable to programmable automated assessment were incorporated into the Editor. A set of natural language processing methods were also adapted for use in the SHeLL Editor, though the approach was primarily procedural (rule-based). As a result of this process, the Editor comprises 6 assessments: readability (school grade reading score calculated using the Simple Measure of Gobbledygook (SMOG)), complex language (percentage of the text that contains public health thesaurus entries, words that are uncommon in English, or acronyms), passive voice, text structure (eg, use of long paragraphs), lexical density and diversity, and person-centered language. These are presented as global scores, with additional, more specific feedback flagged in the text itself. Feedback is provided in real-time so that users can iteratively revise and improve the text. The design also includes a “text preparation” mode, which allows users to quickly make adjustments to ensure accurate calculation of readability. A hierarchy of assessments also helps users prioritize the most important feedback. Lastly, the Editor has a function that exports the analysis and revised text. The SHeLL Health Literacy Editor is a new tool that can help improve the quality and safety of written health information. It provides objective, immediate feedback on a range of factors, complementing readability with other less widely used but important objective assessments such as complex and person-centered language. It can be used as a scalable intervention to support the uptake of health literacy guidelines by health services and providers of health information. This early prototype can be further refined by expanding the thesaurus and leveraging new machine learning methods for assessing the complexity of the written text. User-testing with health professionals is needed before evaluating the Editor’s ability to improve the health literacy of written health information and evaluating its implementation into existing Australian health services.

Development of a website support intervention for families after sudden cardiac death

“…The Flesch-Kincaid Grade level score in Microsoft word was used to assess readability. Pages were also reviewed using the health literacy editor (https://shell.techlab.works/) which helps identify complex words and sentences within text [9].…”

“…It is a widely used and established tool which gives an indication of the school grade reading level needed to read a given text [7, 8]. In general, it is recommended that authors aim for their health information to be readable at a Grade 8 level (13-15 year olds) [9].…”

Australian Women’s Intentions and Psychological Outcomes Related to Breast Density Notification and Information

“…Adaptation involved the use of simpler language, removing jargon, reducing sentence length and complexity, and use of the active voice. This process was achieved using the publicly available Sydney Health Literacy Lab Health Literacy Editor . The letter had a readability level of grade 7.8 on the Microsoft Word Flesch-Kincaid test (with grade ≤8 recommended for average readers).…”

“…Health Literacy Lab Health Literacy Editor. 24 The letter had a readability level of grade 7.8 on the Microsoft Word Flesch-Kincaid test (with grade Յ8 recommended for average readers).…”

The Potential for a Chat-Based Artificial Intelligence Model to Facilitate Educational Messaging on Hypertension

“…Second, we developed another set of 15 questions with different wording but similar meaning in May 2023 to understand ChatGPTs performance between different prompts.Readability of ChatGPT responses was based on the Simple Measure of Gobbledygook Index using the Sydney health literacy lab editor. 2 The Simple Measure of Gobbledygook index is validated in health care information in English language as a measure that estimates the number of years of formal education that a reader would need to understand the information. Health education resources should have a reading grade level of 8 or lower.…”