Seeing the “Big” Picture: Big Data Methods for Exploring Relationships Between Usage, Language, and Outcome in Internet Intervention Data

Abstract

BackgroundAssessing the efficacy of Internet interventions that are already in the market introduces both challenges and opportunities. While vast, often unprecedented amounts of data may be available (hundreds of thousands, and sometimes millions of participants with high dimensions of assessed variables), the data are observational in nature, are partly unstructured (eg, free text, images, sensor data), do not include a natural control group to be used for comparison, and typically exhibit high attrition rates. New approaches are therefore needed to use these existing data and derive new insights that can augment traditional smaller-group randomized controlled trials.ObjectiveOur objective was to demonstrate how emerging big data approaches can help explore questions about the effectiveness and process of an Internet well-being intervention.MethodsWe drew data from the user base of a well-being website and app called Happify. To explore effectiveness, multilevel models focusing on within-person variation explored whether greater usage predicted higher well-being in a sample of 152,747 users. In addition, to explore the underlying processes that accompany improvement, we analyzed language for 10,818 users who had a sufficient volume of free-text response and timespan of platform usage. A topic model constructed from this free text provided language-based correlates of individual user improvement in outcome measures, providing insights into the beneficial underlying processes experienced by users.ResultsOn a measure of positive emotion, the average user improved 1.38 points per week (SE 0.01, t122,455=113.60, P<.001, 95% CI 1.36–1.41), about a 27% increase over 8 weeks. Within a given individual user, more usage predicted more positive emotion and less usage predicted less positive emotion (estimate 0.09, SE 0.01, t6047=9.15, P=.001, 95% CI .07–.12). This estimate predicted that a given user would report positive emotion 1.26 points higher after a 2-week period when they used Happify daily than during a week when they didn’t use it at all. Among highly engaged users, 200 automatically clustered topics showed a significant (corrected P<.001) effect on change in well-being over time, illustrating which topics may be more beneficial than others when engaging with the interventions. In particular, topics that are related to addressing negative thoughts and feelings were correlated with improvement over time.ConclusionsUsing observational analyses on naturalistic big data, we can explore the relationship between usage and well-being among people using an Internet well-being intervention and provide new insights into the underlying mechanisms that accompany it. By leveraging big data to power these new types of analyses, we can explore the workings of an intervention from new angles, and harness the insights that surface to feed back into the intervention and improve it further in the future.

GET.HAPPY2 – User perspectives on an internet‐based self‐management positive psychology intervention among persons with and without depression: Results from a retrospective survey

“…A cluster analysis of the sample revealed two distinct groups of participants – a nondistressed group with an average depression score below the cut‐off (50.5% of the sample) and a distressed group with a depression score well above the cut‐off (49.5% of the sample). Similar results were reported by both Sergeant and Mongrain () and Carpenter et al (). These findings indicate the high relevance of research on the use of PPIs among persons with mental illness, especially among individuals with depression.…”

Engagement with a cognitive behavioural therapy mobile phone app predicts changes in mental health and wellbeing: MoodMission

“…To measure the real-world effects of using MHapps outside of clinical trials, some have suggested investigating and quantifying positive relationships between app usage and mental health outcomes (Carpenter et al, 2016;Yeager & Benight, 2018). This can help avoid "digital placebo effects," which occur when merely the installation of a MHapp can bias individuals' responses towards favourable mental health outcomes (Torous & Firth, 2016).…”

People’s Motives to Participate in a Positive Psychology Intervention with Email Support and Who Might Benefit Most?

“…To be able to compare email counseling in different interventions and/or different study populations, automatic text coding of such large amounts of text data might be beneficial. Different text processing methods exist, such as the software program Linguistic Inquiry and Word Count (LIWC), the latent Dirichlet allocation technique (LDA) (Carpenter et al 2016) and word clouds analysis (McNaught and Lam 2010). However, these methods or programs only detect single words rather than its context (Tausczik and Pennebaker 2010).…”