The Impact of Validity Screening on Associations Between Self-Reports of Bullying Victimization and Student Outcomes

Abstract

Self-report surveys are widely used to measure adolescent risk behavior and academic adjustment, with results having an impact on national policy, assessment of school quality, and evaluation of school interventions. However, data obtained from self-reports can be distorted when adolescents intentionally provide inaccurate or careless responses. The current study illustrates the problem of invalid respondents in a sample ( = 52,012) from 323 high schools that responded to a statewide assessment of school climate. Two approaches for identifying invalid respondents were applied, and contrasts between the valid and invalid responses revealed differences in means, prevalence rates of student adjustment, and associations among reports of bullying victimization and student adjustment outcomes. The results lend additional support for the need to screen for invalid responders in adolescent samples.

School Teasing and Bullying After the Presidential Election

“…Second, surveys that were completed in an unreasonably short amount of time (i.e., ~ 6 minutes), based on the minimum times of survey testers who attempted to complete the survey as quickly as possible, were excluded (i.e., 0.3%–1.1% of respondents). This validity screening procedure has been shown to improve the overall quality of adolescent survey data (Cornell, Klein, Konold, & Huang, 2012), including bullying victimization rates (Jia, Konold, Cornell, & Huang, 2018). Inattentive or careless responders in Internet-based surveys have been shown to reduce reliability estimates (Johnson, 2005); respondents on the low end of completion time distribution can be considered careless responders (Meade & Bartholomew, 2012).…”

Confirmatory Factor Structure and Predictive Validity of the Early Identification System—Student Report in a Community Sample of High School Students

“…Factor analyses were performed using Mplus 6.1 (Muthen & Muthen, 2011 ). Prior to analysis, validity checks showed all responders suggested they had answered all of the items truthfully ( n = 4951; 94%), a check shown to improve the quality of surveys (Cornell, Klein, Konold, & Huang, 2012 ; Furlong, Fullchange, & Dowdy, 2016 ; Jia, Konold, Cornell, & Huang, 2018 ). Analyses were conducted over three phases.…”

A Multilevel MTMM Approach to Estimating the Influences of Contextual Factors on Trait and Informant-Based Method Effects in Assessments of School Climate

“…Students were also removed (an additional 6.4%) from the sample for not being truthful as revealed through responses to two validity questions (i.e., “I am telling the truth on this survey” and “How many of the questions on this survey did you answer truthfully?”) that were not asked of teachers. Failure to screen samples through the use of completion time and validity items has recently been shown to have adverse impacts on the validity of survey data (Jia, Konold, Cornell, & Huang, 2016). Finally, the sample was further restricted to schools with a minimum of five valid teacher respondents (see Konold et al, 2014, for additional technical information and description of sampling procedures).…”

“…that were not asked of teachers. Failure to screen samples through the use of completion time and validity items has recently been shown to have adverse impacts on the validity of survey data (Jia, Konold, Cornell, & Huang, 2016). Finally, the sample was further restricted to schools with a minimum of five valid teacher respondents (see Konold et al, 2014, for additional technical information and description of sampling procedures).…”