The Value of Social Media Language for the Assessment of Wellbeing: A Systematic Review and Meta-Analysis

Abstract

Wellbeing is an important concept that concerns researchers, policy makers, and the broader general public. The measurement of individuals’ wellbeing levels has predominantly been done through self-reports (e.g., survey questionnaires), which is time-consuming for respondents and costly. Alternatively, wellbeing can be measured in real-time by automatically analysing the language expressed on social media platforms (e.g., Facebook, Twitter, Weibo), through social media language text mining (SMTM). The application of this method for the measurement of wellbeing is relatively new, therefore the validity of SMTM for wellbeing is still being established. We present a systematic review based on 45 studies, and a meta-analysis on 32 effect sizes from a subset of 18 studies reporting correlations between SMTM wellbeing and survey-based ground truth measures. Our qualitative synthesis of the reviewed studies provided insights into current patterns in the literature including (1) most studies were conducted in English speaking samples, (2) Twitter was the most popular social media platform for data collection, (3) closed vocabulary dictionary methods driven and word-level methods of analysis were equally preferred, (5) satisfaction with life was the most popular ground-truth measure across the studies. In addition to this, our qualitative synthesis provided support for the face validity of SMTM for wellbeing by comparing/highlighting the similarities between the broader survey-based wellbeing literature and the findings of the SMTM-based wellbeing studies. Our meta-analysis found that SMTM shows convergent validity with traditional wellbeing measures (r = .54, 95% CI [.37, .67] for location level studies, and r = .33, 95% CI [.25, .40] for individual-level assessments).SMTM is a promising and growing method, but researchers should be aware of its current pitfalls such as the non-representativeness of the samples acquired through social media platforms. We provide recommendations for future SMTM studies in the context of wellbeing.

Comparison of wellbeing structures based on survey responses and social media language: A network analysis

“…Additionally, they can reveal the degree and manner in which SMTM‐based wellbeing assessments are similar to existing survey measures. Such a fine‐grained comparison (through the networks) between the survey and SMTM‐based wellbeing is important as the correlational convergence shown by the existing literature (Sametoglu et al, 2022) is only based on the aggregate measures of wellbeing (sum of survey items and the predicted wellbeing levels based on all topics features) and does not allow for a comparison between the individual elements (items and language topics) that adds up to overall survey‐ and SMTM‐based wellbeing scores. Therefore, the present study leverages network analysis to provide a detailed comparison between the two methods.…”

“…This was because the conceptual breadth of the wellbeing survey measures previously used to correlate with SMTM‐based wellbeing scores were mostly limited to hedonic measures of wellbeing (i.e. positive affect or life satisfaction) (Sametoglu et al, 2022). However, contemporary theories of wellbeing (Diener et al, 1985; Keyes et al, 2010; Ryff, 1989) suggest that including both hedonic and eudaimonic wellbeing measures ensure a more comprehensive and detailed picture of an individual's wellbeing levels.…”

“…SMTM methods can provide an unobtrusive and real‐time measurement of wellbeing based on the language coming from the social media accounts of individuals. It is known that survey‐ and SMTM‐based wellbeing correlate with each other (meta‐analytic estimates of .54, 95% CI [.37, .67] for location‐level studies, and .33, 95% CI [.25, .40]) for individual‐level assessments (Sametoglu et al, 2022). Such level of convergence obtained between the survey‐ and SMTM‐based wellbeing scores are considered as an indicator of (convergent) validity.…”

From status updates to states of mind: analyzing parental well-being in the digital landscape

“…Computational development psychology (Shultz, 2003) has started addressing this gap, identifying markers for emotional and psychological states through text analysis of social media posts (Alharthi et al, 2017;Suh et al, 2021), but the specific focus on parents remains understudied (Westrupp et al, 2022). Furthermore, research (Sametoğlu et al, 2023) has shown that results of research using of social media text mining (SMTM) to assess individuals' well-being correlated with results of studies utilizing self-report measures. Thus, social media posts can be a valid source of data for assessment of parental well-being.…”

Beyond the Picture: An Analysis of Emotional Attachment in Relation to Meaningful Natural Spaces.

“…Insights could be drawn about how spatial features of places might influence emotional reactions in an online space. Wellbeing can also be measured through social media data text mining, with strong face validity (compared to selfreport data) and convergent validity reported in studies using this type of data (Chen et al, 2014;Sametoglu et al, 2022). Further, social media platforms have a proven ability to change the way people engage with their surroundings and interact with the environment (Gatti & Procentese, 2021), meaning a study on the discourse of natural spaces could draw insights into how different natural environments are being used.…”