ReHumanizing Hospital Satisfaction Data: Text Analysis, the Lifeworld, and Contesting Stakeholders’ Beliefs in Evidence

Abstract

Declining clinician engagement, increasing rates of burnout, and stagnant patient and family experience scores have led hospital leadership at Seattle Children's Hospital to submit requests to a data scientist and an anthropologist to identify key themes of survey comments and provide recommendations to improve experience and satisfaction. This study explored ways of understanding satisfaction as well as analytic approaches to textual data, and found that various modes of evidence, while seemingly ideal to leaders, are hard pressed to meet their expectations. Examining satisfaction survey comments via text mining, content analysis, and ethnographic investigation uncovered several specific challenges to stakeholder requests for actionable insights. Despite its hype, text mining struggled to identify actionable themes, accurate sentiment, or group distinctions that are readily identified by both content analysis and end users, while more insightful ethnographic results were sometimes discounted for lack of quantitative results or perceived implementation difficulty. Unfortunately, institutional contexts and preferences for specific types of data can lead to unnecessary requests and wasted efforts. Through including the subjective lifeworld, how an individual's lived experiences impact interactions with others, the authors were able re‐humanize satisfaction. Cross‐discipline collaboration can enhance the quality, validation, and advocacy of evidence from both qualitative and quantitative data. Co‐developing a “Return on Method” (ROM) of satisfaction data can help improve analytic requests and expectations by end users. Ultimately, a lifeworld‐informed combination of data science and ethnography can provide contextual and culturally situated insights that are both meaningful and actionable.

Tasks as needs: reframing the paradigm of clinical natural language processing research for real-world decision support

“…Thus, cNLP is typically used for a select set of simplified “what” questions, but not the more complex, and clinically useful “how” or “why” questions. 59 Additionally, cNLP models usually produce binary outputs (presence/absence, true/false) for one or more medical variables often without considering how they meaningfully associate with other medical conditions or events. Several reviews of clinical information extraction applications have found that the vast majority of cNLP models involved an attempt to automatically detect the presence or absence of a disease or injury, adverse medical or treatment events, or patient characteristics, with a small proportion also extracting numeric values from narrative text.…”

“…Rather, to ensure adoptability, researchers need to shift to addressing how reliably the system answers “why” and “how” questions. 59 …”

Collecting and communicating perishable data in a post-disaster context: rapid research and rapid dissemination

“…We transcribed 39% of interviews and conducted language analysis. Findings from the computerassisted language analysis were triangulated with ethnographic observations and direct text analysis (Wignall and Barry, 2018) exploring tensions and contradictions, needs and agency. A careful review of text-based content surrounding top codes from the predetermined list and participant voice frequently used lists generated a third list exploring conceptual relationships between the two.…”