Stop Interrupting Me! Examining the Relationship Between Interruptions, Test Performance and Reactions

Self-selection as an explanation for general mental ability test score differences between mobile and nonmobile devices in observational studies.

“…These findings are important given the growing proportion of mobile device use for testing among job applicants. The overall rate of mobile device usage in our operational sample (30.6%) was 15 times greater than estimates reported by Arthur et al (2014; approximately 2% mobile users) only 6 years ago and nearly double the rate observed by Lawrence et al 3 years ago (2017; approximately 16% mobile users). Not only are mobile-friendly assessments attractive to hiring organizations, but they are also reported to yield more favorable applicant reactions and more diverse applicant pools (Woods et al, 2020).…”

“…However, their experimental study did not result in any differences in test scores between device types, even when mobile devices were used outdoors and distractions were more commonly reported. Likewise, in a correlational study of 5,549 job applicants, Lawrence et al (2017) reported that nearly all mobile job applicants reported completing their tests while at home (96%) and were more likely to test at home compared to nonmobile users. This contradicts the idea that mobile test-takers are more likely to test in public or more distracting environments just because mobile technology allows them to do so.…”

The testing environment as an explanation for unproctored internet‐based testing device‐type effects

“…For instance, in an examination of the relationship between interruptions and test performance and UITs in operational settings, Lawrence et al. (2017) reported that a wide range of interruptions and distractions (e.g., receiving pop‐ups, push notifications, emails, text messages, phone calls, technical issues, being directly spoken to, having people around, background noises) were reported by test takers completing assessments on both mobile and nonmobile devices. In addition, their results showed that the device type moderated the relationship between interruptions and test performance such that, counterintuitively, nonmobile test takers' scores were more influenced by interruptions than mobile test takers' scores.…”

“…However, in operational settings or contexts, to the extent that UITs grant the test taker vast degrees of freedom in terms of when, how (i.e., device), and where (i.e., location) to complete them, the environment wherein one completes a UIT may be less than ideal for optimal performance. That is, the greater degrees of freedom in where one can complete the assessment generally covaries with a greater potential for distractions in the testing environment (Gray et al., 2015; Lawrence et al., 2017). Indeed, Gray et al.…”