Effort in Low-Stakes Assessments

Abstract

Performance of students in low-stakes testing situations has been a concern and focus of recent research. However, researchers who have examined the effect of stakes on performance have not been able to compare low-stakes performance to truly high-stakes performance of the same students. Results of such a comparison are reported in this article. GRE test takers volunteered to take an additional low-stakes test, of either verbal or quantitative reasoning as part of a research study immediately following their operational high-stakes test. Analyses of performance under the high- and low-stakes situations revealed that the level of effort in the low-stakes situation (as measured by the amount of time on task) strongly predicted the stakes effect on performance (difference between test scores in low- and high-stakes situations). Moreover, the stakes effect virtually disappeared for participants who spent at least one-third of the allotted time in the low-stakes situation. For this group of test takers (more than 80% of the total sample), the correlations between the low- and high-stakes scores approached the upper bound possible considering the reliability of the test.

Test-Taking Motivation in Education Students: Task Battery Order Affected Within-Test-Taker Effort and Importance

“…Moreover, Penk and Richter (2017) identified changes in the motivational component of test-taking effort during test taking. Test takers' self-reported test-taking effort decreased from the beginning to the end of the test in this study (Penk and Richter, 2017) and in other studies (Attali, 2016;Lindner et al, 2018). Test takers may easily recognize that the TTMQ items are intended to capture their motivational state and might thus respond in socially desirable ways.…”

“…Test-taking effort has been investigated by different measures, for example, response times (Wise and Kong, 2005), time on task (Attali, 2016), or self-reports (Knekta and Eklöf, 2015). A study compared test-taking effort (measured by time on task) and performance in a high-stakes achievement test vs. subsequent low-stakes achievement test with the result that the majority of test takers replicated their high stakes performance in the low-stakes condition with little effort (Attali, 2016).…”

Performing up to par? Performance pressure increases undergraduates’ cognitive performance and effort.

“…Finally, our mediator of interest was participants' exerted effort on the WM task, which we measured using response time during the WM task. Consistent with prior work using time-on-task to measure effort (e.g., Attali, 2016), we anticipated that slower response times would indicate greater exerted effort (Bonner & Sprinkle, 2002). We manipulated feedback on the WM task across studies.…”

“…accounted for much of the gap in scores (Attali, 2016). Indeed, researchers in educational assessment recommend using response times as an index for effort, with exceedingly quick response times indicating unmotivated, noneffortful performance (see Wise & DeMars, 2010;Wise & Kong, 2005).…”

“…One explanation for the benefits of gamification may be affective, such that uncertainty creates enjoyment which is arousing and increases effort. Similarly, pressure, in the form of performance-contingent rewards or raising the stakes, has been shown to incentivize performance on a variety of tasks, including standardized tests (e.g., graduate record exam [GRE]; Attali, 2016; Schlosser et al, 2019) and working memory (WM) tasks (e.g., Heitz et al, 2008; Jimura et al, 2010). Yet, pressure can also threaten performance by inducing verbally rehearsed worries that co-opt those very WM resources (see Beilock, 2008).…”