Computer Assisted Tailored Testing: Examinee Reactions and Evaluations

Abstract

In the last few years, computer assisted tailored testing has been shown to be technically feasible; the question of its acceptability to examinees therefore assumes increased importance. This report presents the reactions and opinions of 163 examinees who participated in a tailored testing pilot study conducted at the U. S. Civil Service Commission during the fall of 1975. The reactions of the examinees can accurately be summarized as overwhelmingly positive, thus disposing of yet another potential obstacle to the advance of this innovative technique.

Reactions to Computerized Testing in Selection Contexts

“…Previous research has shown not only positive overall reactions from test-takers towards computerized testing (Barbera et al, 1995;Schmidt et al, 1978;Slack and Van Cura, 1968), but also that test-takers prefer computerized versions of tests over paper-and-pencil versions (Arvey et al, 1990;Ogilvie, Trusk and Blue, 1999;Richman-Hirsch et al, 2000). These findings may be attributed to the novelty of the computerized testing.…”

“…Researchers have found that applicants either rate different administration modes of a test as equally fair (Daum, 1994) or rate the computerized version as more fair (e.g., Schmidt et al, 1978;Schmitt et al, 1993) than the traditional paper-and-pencil version. This may be due to people's perceptions that computers are more objective, accurate, and less prone to biases than traditional forms of selection testing.…”

“…The current study examines the antecedents and outcomes associated with reactions to computerized testing procedures in a selection setting. Fourth, most of the research on reactions to computerized testing has used either cognitive ability tests (e.g., Arvey et al, 1990;Burke et al, 1987;Daum, 1994;Landis, Davison and Maraist, 1998;Schmidt, Urry and Gugel, 1978) or interviewing (e.g., Lucas et al, 1977;Martin and Nagao, 1989;O'Brien and Dugdale, 1978). There is limited research using other types of computerized tests such as multimedia tests (Barbera, Ryan, Burris and Dyer, 1995;Richman-Hirsch, Olson-Buchanan and Drasgow, 2000;Shotland and Alliger, 1999) and video-based tests ) that go beyond simply transferring a paper-and-pencil test to a computer.…”

An Empirical Study of the Broad Range Tailored Test of Verbal Ability

“…found that for high-ability students, motivation was high on both stradaptive and conventional tests administered on a cathode ray terminal, while for low ability students it was high only on the stradaptive test. Schmidt, Urry, and Gugel (1977) of those responding indicated a preference for pencil-and-paper testing, and 4% felt that the adaptive test was less fair than a conventional test.…”

“…The features least liked were: the inability to review and change previous answers (23%) and difficulty in adjusting to this method of administration (20%) (Schmidt, Urry, and Gugel, 1977 Research on KR in an adaptive testing environment has been conducted by Betz and Weiss (l976a" h). Pine (1977), andPrestwood (1977).…”