Seriousness checks are useful to improve data validity in online research

Abstract

Nonserious answering behavior increases noise and reduces experimental power; it is therefore one of the most important threats to the validity of online research. A simple way to address the problem is to ask respondents about the seriousness of their participation and to exclude self-declared nonserious participants from analysis. To validate this approach, a survey was conducted in the week prior to the German 2009 federal election to the Bundestag. Serious participants answered a number of attitudinal and behavioral questions in a more consistent and predictively valid manner than did nonserious participants. We therefore recommend routinely employing seriousness checks in online surveys to improve data validity.

Investigating measurement equivalence of visual analogue scales and Likert-type scales in Internet-based personality questionnaires

“…Due to the research design requiring variance in the age of participants, they were instructed to recruit participants of varying ages to counter the potential problem of range restriction in this dependent measure. The questionnaire contained the seriousness check (Aust, Diedenhofen, Ullrich, & Musch, 2013;Reips, 2002Reips, , 2009) at the beginning. Participants then indicated their age and gender, before the personality scales were presented.…”

The communality-bonus effect for male transformational leaders – leadership style, gender, and promotability

“…", "I have never used a computer. "), they were excluded (Aust, Diedenhofen, Ullrich, & Musch, 2013). At Time 1, we asked participants to rate a member of their team.…”

An Internet Resource for Self-Assessment of Mental Health and Health Behavior: Development and Implementation of the Self-Assessment Kiosk (Preprint)

“…measures of physical and emotional well-being at the time of the survey (visual analog scales from 0 to 100) and a seriousness check, which has been shown to increase the validity of online self-reports [29]. The serious check used was the following: "Sometimes people 'check out' the Self-Assessment Kiosk without providing information that is true about them.…”