Detecting Insufficient Effort Responding with an Infrequency Scale: Evaluating Validity and Participant Reactions

Dirty Data: The Effects of Screening Respondents Who Provide Low-Quality Data in Survey Research

“…There are three types of items that can be included in a survey to directly evaluate data quality, including self-reported effort items (e.g., "I carefully considered each item before responding"; Berry et al, 1992;Costa & McCrae, 1997), "bogus items" (e.g., "I was born on February 30"; Bagby, Gillis, & Rogers, 1991;Huang, Bowling, Liu, & Li, 2015), and "instructed items (e.g., "Please mark 'slightly agree' for this item"). Participants are flagged as potentially providing LQD if they indicate their responses are untrustworthy, provide illogical responses to bogus items, or fail to follow instructions in instructed items.…”

Doing online research involving university students with disabilities: Methodological issues

“…Moreover, it can be expected that biases will be particularly present in the responses of university students with disability, due to the explicit will to deliver a positive self-image to protect oneself from stigma (Frable et al, 1998;Goffman, 1963); this may, in turn, depend on the survey mode, as previous reports have indicated higher disclosure of sensitive information in computerized or online compared to paper-and-pencil questionnaires Crocker & Major, 1989;Gnambs & Kaspar, 2014;Joinson et al, 2010). While there are methods and measures to examine several forms of response distortion (e.g., careless responding, or social desirability), it must be noted that all these measure constitute attempts, which are performed after the questionnaire has been completed, to detect or eliminate the impact of non-observable factors (like carelessness or social desirability; Connelly & Chang, 2015;de Vries et al, 2014;Dodou & de Winter, 2014;Huang et al, 2012Huang et al, , 2015Paulhus, 2002;Paunonen & LeBel, 2012). However, carrying out post hoc analyses can only provide an estimation of whether a response was careless, or socially desirable, while the real status of the response (careless or not, biased or from a conscientious individual) remains unknown.…”

“…These indexes may be based on the time used to respond, on the detection of infrequent response sequences (such as a long list of identical responses), or the detection of wrong, odd or inconsistent responses (such as agreeing both with ''I love my job'' and ''I hate my job''). These measures of IER are related to each other (Huang, Bowling, Liu, & Li, 2015;Huang, Curran, Keeney, Poposki, & DeShon, 2012), and removing participant data which are suspected of IER may improve the psychometric properties of a questionnaire (Huang et al, 2012). How can measures of IER be implemented in an online study?…”

“…Some measures of IER (e.g., response time and infrequent responses) are unrelated to the specific questionnaire which is being filled out, and concern the time used to respond (response time 1 ) or the responses to filler questions which have a clear correct response (e.g., responding ''yes'' to ''I was born on July 38th''). On the other hand, other measures of IER (e.g., psychometric antonyms and individual reliability; Huang et al, 2015;Johnson, 2005;Meade & Craig, 2012) depend on the responses given to each question in the survey, and more specifically are based on the calculation of individual correlation between a priori defined groups of questions (individual reliability) or psychometrically defined groups of questions (psychometric antonyms). In the case of the WHOQOL, the questions reflect general and specific subdomains of quality of life, and each of these subdomains contains multiple questions.…”

Public versus Private Voice

“…Could it be that an employee's perceived competence, warmth, positive dominance, and negative dominance depend, all else being equal, on whether the employee raises voice in public versus private? To find out, I recruited 470 undergraduate students from a Dutch university of which 465 participants (51% female, 58% German, 16% Dutch, Mage = 20.04) provided reliable data (5 participants needed to be excluded because of a response pattern known to indicate insufficient effort; Huang, Bowling, Liu, & Li, 2015). I adapted a scenario study from Sijbom and colleagues (2015b) asking participants to imagine that they were the head of a marketing department in a company specializing in frozen fast food, responsible for planning the marketing strategy of the company, and managing the work of the department's 12 employees.…”