The relationships between selected subtests from the Armed Services Vocational Aptitude Battery (ASVAB) and corresponding subtests administered as computerized adaptive tests (CAT) were investigated using Marine recruits as subjects. Three adaptive subtests were shown to correlate as well with ASVAB as did a second administration of ASVAB, even though the CAT subtests contained only half the number of items. Factor analysis showed the CAT subtests to load on the same factors as the corresponding ASVAB subtests, indicating that the same abilities were being measured. The preenlistment Armed Forces Qualification Test (AFQT) composite scores were predicted as well from the CAT subtest scores as from the retest ASVAB subtest scores, even though the CAT contained only three of the four AFQT subtests. It is concluded that CAT can achieve the same measurement precision as a conventional test, with half the number of items.The Department of Defense is currently developing a computerized adaptive testing (CAT) system as a potential replacement for the conventional paper-and-pencil tests used for enlisted personnel selection and classification. The existing Armed Services Vocational Aptitude Battery (ASVAB) consists of a fixed sequence of test items administered to all examinees. CAT entails automated tailoring of a sequence of test items to each ex-APPLIED PSYCHOLOGICAL MEASUREMENT aminee, contingent upon his/her responses to earlier items in the sequence (e.g., Weiss, 1982Weiss, , 1983. Correct responses are generally followed by more difficult items and incorrect responses by easier items. CAT requires substantially fewer test items than a conventional test because items that are too easy or too difficult for the examinee are not administered. Additionally, computerization offers further advantages by eliminating the clerical errors inherent in manual test administration and by increasing test security.In developing CAT as a replacement for ASVAB, care is being taken to ensure that CAT tests will be as accurate as the current printed ASVAB tests. Data related to this question have been presented by McBride (1980) and by McBride and Martin (1983), who found that an adaptive test of verbal ability was more reliable and more valid than a conventional test. Concern for CAT's accuracy also raises the question of whether CAT and ASVAB measure the same abilities. A limited amount of research has been done on the relationship between CAT and the conventional tests currently employed in the military (Sympson, Weiss, & Ree, 1982). Further investigation is needed.The objective of this study was to determine (1) the relationship between scores on selected paperand-pencil ASVAB subtests and an experimental battery of three corresponding CAT subtests, and (2) whether corresponding CAT and ASVAB subtests measure the same aptitudes.

This study sought to determine whether the mean score on the computerized version of an arithmetic reasoning test would be significantly lower than that on the paper-and-pencil version when there was no time limit. It was hypothesized that there would be no significant difference between the mean scores. A test for an interaction between Mode of Administration and ability was also performed. Test performance was measured by the number-correct score on the Experimental Arithmetic Reasoning Subtest; ability was measured by the Arithmetic Reasoning subtest of the Armed Services Vocational Aptitude Battery. Subjects were 585 military recruits between the ages of 18 and 25. There were 300 subjects in the Paper-and-Pencil Mode and 285 in the Computer Mode. Subjects were randomly assigned to either mode. A significant main effect for Mode (p < .05) was found, with the mean score obtained by computer lower than that obtained by paper-and-pencil. No interaction between Mode and ability was found. Possible explanations are given and discussed.

By the mid-l980s, an item pool had been constructed for use in the experimental CAT-ASVAB system (Chapter 9), and had been administered to a large number of subjects participating in research studies. However, this pool was ill-suited for operational use. First, many items had been taken from retired P&P-ASVAB forms (€49, and 10). Using these items in an operational CAT-ASVAB would degrade test security, since these items had broad exposure through the P&P testing program. In addition, the experimental CAT-ASVAB system contained only one form. For retesting purposes, it is desirable to have two parallel forms (consisting of non-overlapping item pools) to accommodate applicants who take the battery twice within a short time interval. To avoid practice and compromise effects, it is desirable for the second administered form to contain no common items with the initial form.This chapter summarizes the procedures used to construct and evaluate the operational CAT-ASVAB item pools. Although specific reference is made to Forms 1 and 2, many of the same procedures were applied more recently to the development of other CAT-ASVAB forms. The first section describes the development of the primary and supplemental item banks. Additional sections discuss dimensionality, alternate form construction, and precision analyses. The final section summarizes important findings with general implications for CAT item pool development. Development and Calibration Primary Item BanksThe primary item banks for CAT-ASVAB Forms 1 and 2 were developed and calibrated by Prestwood, Vale, Massey, and Welsh (1985). The P&P-ASVAB Form 8A was used to outline the content of items written in each area. However, important differences between the development of adaptive and conventional (paper-and-pencil) item pools were noted, which led to several modifications in P&P-ASVAB test specifications:Increased range of item dificulties Domain specifications were expanded to provide additional easy and difficult items. 9 Functionally independent items The Paragraph Comprehension test (as measured in P&P-ASVAB) typically contains reading passages followed by several questions referring to the same passage. Items of these types are likely to violate the assumption of local independence made by the standard unidimensional IRT model. Consequently, CAT-ASVAB items were written to have a single question per passage.

No abstract

No abstract