We provide a psychometric analysis of commonly used performance indices of the d2 sustained-attention test, and give methodical guidelines and recommendations, based on this research. We examined experimental effects of repeated testing on performance speed and accuracy (omission and commission errors), and further evaluated aspects of test reliability by means of cumulative reliability function (CRF) analysis. These aspects were also examined for a number of alternative (yet commonly used) scoring techniques and valuation methods. Results indicate that performance is sensitive to change, both differentially within (time-on-task) and between (test-retest) sessions. These effects did not severely affect test reliability, since perfect score reliability was observed for measures of speed (and was even preserved with half the test length) while variability and error scores were more problematic with respect to reliability. Notably, limitations particularly hold for commission but less so for omission errors. Our recommendations to researchers and practitioners are that (a) only the speed score (and error-corrected speed score) is eligible for highly reliable assessment, that (b) error scores might be used as a secondary measure (e.g., to check for aberrant behavior), that (c) variability scores might not be used at all. Given the exceptional reliability of performance speed, and (d) test length may be reduced up to 50%, if necessary for time-economic reasons, to serve purposes of population screening and field assessment. (PsycINFO Database Record

Reaction time is typically increased following an erroneous response. This post-error slowing is traditionally explained by a strategic adjustment of response threshold towards more conservative behaviour. A recently proposed orienting account provides an alternative explanation for post-error slowing. According to this account, committing an error evokes an orienting response (OR), which inhibits information processing in the subsequent trial, resulting in slow and inaccurate performance. We tested a straightforward prediction of the orienting account in the context of self-paced performance, adopting an individual-differences approach: Post-error slowing should be larger the less frequent an error is. To this end, participants were classified into three groups differing in overall performance accuracy. Larger post-error slowing and stronger post-error accuracy decrease were observed for the high-accuracy group than for the two other groups. Practice pronounced the post-error accuracy decline, especially for the high-accuracy group. The results are consistent with the orienting account of post-error slowing but are problematic for accounts based on strategic evaluation mechanisms.

SummaryThe goal of this study is to understand the interface between subjectively assessed and objectively measured attention functioning. To this end, we assessed both self‐report data using the Cognitive Failures Questionnaire (CFQ) and sustained‐attention performance using a low‐demand psychomotor vigilance task that required participants to respond to visual targets. Unlike previous studies, reliability of task performance was ensured, testing individuals twice within a retest interval of 1 week. Given that everyday life attentional lapse tendencies vary even in normal‐population samples, we asked whether there is a relationship with laboratory performance, in general and as a consequence of time on task (TOT). As a result, low (versus high) CFQ scorers were somewhat different in their average response speed but became particularly prone to lapsing during the task period, as reflected in standard measures of response‐speed variability as well as ex‐Gaussian parameters of distributional skewness. In conclusion, we argue that persistence to sustained demand might be an important aspect of construct validity that must be evoked by a manipulation of TOT, which may be useful for the evaluation of questionnaires in ecologically valid situations. Copyright © 2015 John Wiley & Sons, Ltd.

Time-related changes in the speeded performance of complex cognitive tasks are considered to arise from the combined effects of practice and mental fatigue. Here we explored the differential contributions of practice and fatigue to performance changes in a self-paced speeded mental addition and comparison task of about 50 min duration, administered twice within one week’s time. Performance measures included average response speed, accuracy, and response speed variability. The results revealed differential effects of prolonged work on different performance indices: Practice effects, being more pronounced in the first session, were reflected in an improvement of average response speed, whereas mental fatigue, occurring in both sessions, was reflected in an increase of response speed variability. This demonstrates that effects of mental fatigue on average speed of performance may be masked by practice effects but still be detectable in the variability of performance. Therefore, besides experimental factors such as the length and complexity of tasks, indices of response speed variability should be taken into consideration when interpreting different aspects of performance in self-paced speed tests.

Previous research suggests a relationship between neuroticism (N) and the speed-accuracy tradeoff in speeded performance: High-N individuals were observed performing less efficiently than low-N individuals and compensatorily overemphasizing response speed at the expense of accuracy. This study examined N-related performance differences in the serial mental addition and comparison task (SMACT) in 99 individuals, comparing several performance measures (i.e., response speed, accuracy, and variability), retest reliability, and practice effects. N was negatively correlated with mean reaction time but positively correlated with error percentage, indicating that high-N individuals tended to be faster but less accurate in their performance than low-N individuals. The strengthening of the relationship after practice demonstrated the reliability of the findings. There was, however, no relationship between N and distractibility (assessed via measures of reaction time variability). Our main findings are in line with the processing efficiency theory, extending the relationship between N and working style to sustained self-paced speeded mental addition.