A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use

Abstract

There is widespread public and academic interest in understanding the uses and effects of digital media. Scholars primarily use self-report measures of the quantity or duration of media use as proxies for more objective measures, but the validity of these self-reports remains unclear. Advancements in data collection techniques have produced a collection of studies indexing both self-reported and log-based measures. To assess the alignment between these measures, we conducted a meta-analysis of this research. Based on 106 effect sizes, we found that self-reported media use only moderately correlates with logged measurements, that self-reports were rarely an accurate reflection of logged media use, and that measures of problematic media use show an even smaller association with usage logs. These findings raise concerns about the validity of findings relying solely on self-reported measures of media use. The materials needed to reproduce the analysis and an article preprint are available at: https://osf.io/dhx48/.

The role of depression in the discrepancy between estimated and actual smartphone use: A cubic response surface analysis.

“…Following the general trend in social psychology of using estimates of behavior as a proxy for actual behavioral measures (Sassenberg & Ditrich, 2019), most studies, however, rely on self-reported measures of DMU (Griffioen et al, 2020). When compared to more objective measures of DMU (i.e., digital trace data or device usage logs), such estimates are generally inaccurate (Parry et al, 2021). Crucially, evidence suggests that the error in self-reported DMU is likely biased systematically by factors that are fundamental to the effect being investigated: Respondents' volume of use (Araujo et al, 2017;Boase & Ling, 2013;Ernala et al, 2020;Scharkow, 2016;Vanden Abeele et al, 2013) and level of depression (Sewall et al, 2020).…”

“…Although self-reported estimates are prevalent in studies of DMU, there is strong evidence that these measures do not capture what they are intended to measure: actual use (Parry et al, 2021). Rather, as is common with self-reports of behavior in many domains (see, e.g., Jenner et al, 2006;Kormos & Gifford, 2014), self-report measures of DMU capture respondents' perceptions of their use rather than their actual use (Scharkow, 2016;Sewall et al, 2020).…”

“…As such, the myriad factors that impact perception-as well as the other cognitive and affective processes that are called upon when estimating DMU-will influence respondents' reports. In this way, self-report estimates of DMU may capture some of the respondent's actual use-as is evident in the moderate correlations found between self-reported and device-logged DMU (Parry et al, 2021)-but also unintentionally capture elements of the respondent's attitudes, perceptions, feelings, cognitions, etc. that are unrelated to actual use (Ellis, 2019).…”

The influence of working-from-home and digital device use on sleep, physical activity, and wellbeing following COVID-19 lockdown and reopening

“…As the pandemic situation has naturally impeded data collection from human subjects, much research has relied on survey data 57 . By combining objective tracking of activity, sleep and phone data, with participants' self-report through ecological momentary assessment, the current study provides longitudinal insights, while minimizing reporting biases associated with (retrospective) surveys responses 14,[58][59][60][61]…”

The Psychology of Mobile Technology and Daily Mobility

“…However, self-reports of mobile media use have faced criticism for being prone to a number of biases. For example, self-reports of constructs that have verifiable answers (e.g., time spent on smartphones) have been found to be fairly inaccurate, if not systematically biased (e.g., confounded with well-being and mental health) (Parry et al, 2021). For self-reports of other constructs (e.g., habitual phone use), it can still be challenging to determine validity, especially when measured in a cross-sectional survey (Ohme, Albaek, & de Vreese, 2016).…”