Integrating Natural Language Processing and Interpretive Thematic Analyses to Gain Human-Centered Design Insights on HIV Mobile Health: Proof-of-Concept Analysis

Abstract

Background HIV mobile health (mHealth) interventions often incorporate interactive peer-to-peer features. The user-generated content (UGC) created by these features can offer valuable design insights by revealing what topics and life events are most salient for participants, which can serve as targets for subsequent interventions. However, unstructured, textual UGC can be difficult to analyze. Interpretive thematic analyses can preserve rich narratives and latent themes but are labor-intensive and therefore scale poorly. Natural language processing (NLP) methods scale more readily but often produce only coarse descriptive results. Recent calls to advance the field have emphasized the untapped potential of combined NLP and qualitative analyses toward advancing user attunement in next-generation mHealth. Objective In this proof-of-concept analysis, we gain human-centered design insights by applying hybrid consecutive NLP-qualitative methods to UGC from an HIV mHealth forum. Methods UGC was extracted from Thrive With Me, a web app intervention for men living with HIV that includes an unstructured peer-to-peer support forum. In Python, topics were modeled by latent Dirichlet allocation. Rule-based sentiment analysis scored interactions by emotional valence. Using a novel ranking standard, the experientially richest and most emotionally polarized segments of UGC were condensed and then analyzed thematically in Dedoose. Design insights were then distilled from these themes. Results The refined topic model detected K=3 topics: A: disease coping; B: social adversities; C: salutations and check-ins. Strong intratopic themes included HIV medication adherence, survivorship, and relationship challenges. Negative UGC often involved strong negative reactions to external media events. Positive UGC often focused on gratitude for survival, well-being, and fellow users’ support. Conclusions With routinization, hybrid NLP-qualitative methods may be viable to rapidly characterize UGC in mHealth environments. Design principles point toward opportunities to align mHealth intervention features with the organically occurring uses captured in these analyses, for example, by foregrounding inspiring personal narratives and expressions of gratitude, or de-emphasizing anger-inducing media.

Leveraging Natural Language Processing to Evaluate Young Adults’ User Experiences with a Digital Sleep Intervention for Alcohol Use

“…User experience evaluations of digital health tools should incorporate a hybrid, convergent approach to optimize depth, breadth, and e ciency. Each user experience evaluation method has distinct strengths and potential weaknesses 23 . Thematic analysis of exit interviews can provide detailed information across complex subjective experiences.…”

“…Each of these attributes increased the validity of ndings. Further, our hybrid, convergent approach incorporating NLP represents an important strength as these methods are promising and relatively new in digital medicine [22][23] . Our research also had notable limitations.…”

“…Further, exit surveys using Likert-type satisfaction scales may be subject to response bias. A hybrid approach combining NLP, quantitative survey analysis, and targeted qualitative interview analysis, may reveal broad and rich user experiences 23 while maximizing researcher time and effort.…”

“…Natural language processing is an umbrella term for a growing suite of machine learning methods that uses arti cial intelligence (AI) to understand (e.g., summarize, retrieve information) and/or generate language content. Traditional evaluation methods, such as qualitative thematic analysis of exit interviews, can reveal nuance and detail, but can be impractical, especially with large sample sizes 23 . Further, exit surveys using Likert-type satisfaction scales may be subject to response bias.…”

A Trauma-Informed, Geospatially Aware, Just-in-Time Adaptive mHealth Intervention to Support Effective Coping Skills Among People Living With HIV in New Orleans: Development and Protocol for a Pilot Randomized Controlled Trial

“…We abstracted the CTSC-based strategies built into NOLA GEM from their original, in-person, session-by-session framework to activate the "just-in-time" aspect of a JITAI adaptation while mitigating ethical concerns that, if faced with an acute stressor, a NOLA GEM user would be denied the best-suited coping support if they had not yet reached the appropriate session had a strictly linear intervention model been retained. Similarly, although a peer-to-peer support forum or social media-style feature [146,147] in which users could interact pseudonymously was briefly considered, lacking active human (vs algorithmic) content moderation, such a space could become consumed with off-topic, inane, or even outrage-inducing user-generated content and external media [86,148]. In addition, by recruiting people living with HIV who were either violence-affected individually or lived in a high-violence neighborhood, a degree of heterogeneity in needs was introduced into the aim-1 and aim-2 samples, which may have swayed the formative insights-and, by extension, intervention design-derived from the New Orleans Alcohol Use in HIV study subsamples.…”