The Effect of Computerized Adaptive Testing on Motivation and Anxiety: A Systematic Review and Meta-Analysis

Abstract

Although many studies have been carried out on the psychometric aspects of computerized adaptive testing (CAT), its psychological aspects are less researched. Early studies claimed that CAT can be more motivating and induce less anxiety than traditional fixed-item tests (FIT). The purpose of this systematic review and meta-analysis was to gain a comprehensive understanding of the effects of CAT on motivation and anxiety in comparison to traditional fixed-item testing. Seven databases were examined. Articles were eligible if they employed an empirical study containing a direct comparison between CAT and FIT. Meta-analytical results showed no overall effect of test type on anxiety and motivation when comparing CAT with FIT ( k = 11, g+ = 0.06, p = .28). However, easier CAT had positive effect compared with FIT ( k = 2, g+ = .22, p < .001). Certain modifications in CAT administration can provide positive psychological effects for test-takers.

Meta-Analysis of the Effects of Computerized Adaptive Testing on the Motivation and Emotion of Examinees

“…While the recently published meta-analysis of Akhtar et al (2023) certainly provides relevant insights, it does not, however, allow for a differentiated understanding of the complex motivational and emotional effects of CAT. The meta-analysis presented in this article aims to provide such a differentiated understanding by extending Akhtar et al (2023) in three different ways. First, it better covers the available empirical studies by including not only articles written in English but also articles written in German and Chinese, as well as unpublished manuscripts if they are of appropriate quality.…”

“…The only moderator variable considered in this meta-analysis, namely, CAT with easier items (in a typical CAT, items are presented that have a probability of around .50 of the individual examinee answering them correctly; this can be altered so that systematically easier items are presented, e.g., with a response probability of .70), had a significant effect on anxiety ( g = 0.22, p = .001). While the recently published meta-analysis of Akhtar et al (2023) certainly provides relevant insights, it does not, however, allow for a differentiated understanding of the complex motivational and emotional effects of CAT. The meta-analysis presented in this article aims to provide such a differentiated understanding by extending Akhtar et al (2023) in three different ways.…”

“…On the basis of existing publications, it is already possible to disentangle the interactions between several (a) test, (b) examinee, and (c) test setting variables for CAT when these variables are used as moderators in a meta-analysis. Until now, only one meta-analysis has focused on the motivational and emotional effects of CAT (Akhtar et al, 2023). This meta-analysis, however, considered only 11 studies and did not include several key studies in this area.…”