Short-form tests: A methodological review.

Levy, Philip

doi:10.1037/h0025736

Cited by 98 publications

(71 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Several psychometric studies investigated properties of short tests (Burisch, 1997;Gardner et al, 1998;Levy, 1968;Smith, McCarthy, & Anderson, 2000;Stanton et al, 2002) but did not take decision quality into consideration. Research in educational testing and clinical assessment addressed decision quality in the context of classification (e.g., pass-fail decisions, assigning treatments), which is formally similar to cut-score selection (Berk, 1986;Emons et al, 2007;Ercikan & Julian, 2002).…”

Section: Introductionmentioning

confidence: 99%

Test Length and Decision Quality in Personnel Selection: When Is Short Too Short?

Kruyen

Emons

Sijtsma

2012

International Journal of Testing

View full text Add to dashboard Cite

Personnel selection shows an enduring need for short stand-alone tests consisting of, say, 5 to 15 items. Despite their efficiency, short tests are more vulnerable to measurement error than longer test versions. Consequently, the question arises to what extent reducing test length deteriorates decision quality due to increased impact of measurement error. A distinction was made between decision quality at the group level and the individual level. Using simulations, we found that short tests had a large negative impact on individual-level decision quality, in particular for selecting suited candidates when base rates or selection ratios are low. Negative effects on group-level decision quality were smaller than those for individual-level decision quality. Results were similar for dichotomous-item tests and rating-scale tests, and also for top-down and cut-score selection.Personnel selection shows the increasing tendency to use short tests consisting of, say, 5 to 15 items, for making decisions about individuals applying for a job (e.g., 322KRUYEN, EMONS, SIJTSMA have been aware of the influence of test length on test-score reliability under idealized conditions of parallel test parts, which are often individual items but in fact any test parts (like subsets of items) that are parallel. The message of the Spearman-Brown prophecy formula is that reliability increases as the test grows longer but the reverse message is that reliability decreases as the test grows shorter. Although somewhat weaker, these trends are also effective if test parts are not parallel, which is common in real tests.Given this century-old knowledge, the trend of using short tests is intriguing, and the question how the use of short and, hence, less-reliable test scores affects the quality of decision-making thus becomes paramount. Emons, Sijtsma, and Meijer (2007) investigated this question for cut-score selection in a computational study and concluded that as test length decreases, the certainty by which one makes the right decisions about individuals can be impaired and become worrisome. Here, we extended their study to include interesting results at the group level in addition to the individual level. Group level is important because hiring many unsuited applicants causes high organizational costs. Individual level needs to be considered because incorrectly selected applicants may gain bad work experiences, while incorrectly rejected applicants may decide to sue the organization for being rejected on the basis of inadequate testing procedures. Assessing group-level and individuallevel decision quality requires different perspectives, as we point out later. We studied the influence of test shortening on decision quality for groups and individuals in five selection scenarios that are regularly discussed in the literature on personnel selection. Our study sheds light on the issue whether short tests can be used responsibly in personnel selection and if so, under which conditions this is possible. BACKGROUND

show abstract

Section: Introductionmentioning

confidence: 99%

Test Length and Decision Quality in Personnel Selection: When Is Short Too Short?

Kruyen

Emons

Sijtsma

2012

International Journal of Testing

View full text Add to dashboard Cite

show abstract

“…Another important issue is the amount of time saved by the use of a short form. Although there is no universally accepted rule for short-form administration time, it has been suggested that an abbreviated scale should reduce testing time by at least 50% (Levy, 1968). As Table 5 indicates, eight of the suggested short forms reduced administration time by more than 50%.…”

Section: Discussionmentioning

confidence: 99%

Administration time estimates for Wechsler Intelligence Scale for Children‐IV subtests, composites, and short forms

2007

View full text Add to dashboard Cite

The administration times for Wechsler Intelligence Scale for Children-IV (WISC-IV) subtests, indexes, and the Full Scale IQ were recorded for 57 school children. Also determined were administration times for eight short forms and the General Ability Index (GAI). All eight short forms reduced testing time by >50%, but the GAI required approximately 56 minutes. The time to administer the 10 core subtests that yield the Full Scale IQ and index scores averaged 72 minutes (range = 42 to 100), but 31% of the administrations required 80 minutes or longer. These results indicate that administration times are quite variable and that D. Wechsler's (2003) guideline of 65 to 80 minutes can be misleading for certain settings and for specific examinees. The present research found administration time to be positively correlated with examinee age, grade placement, and Full Scale IQ. In addition, the extent of examiner experience is known to be positively related to administration speed. In the present research, as in many settings, the examiners were competent, but not highly experienced.

show abstract

“…In a cogent review of short form tests Levy (1968) stated that 'the general requirement that an abbreviated test should save time while retaining maximum validity cannot be satisfied in the absence of a statement about the relative utility of 'time saved' and 'validity lost'. In this connection, although we have not abbreviated a test, we may note that while it is difficult to estimate what validity might be lost by using the three-minute reasoning test instead of the AH4, in terms of the amount of time saved the threeminute test is approximately ten times as quick as the AH4 (and, in addition, it is much easier to score).…”

Section: Discussionmentioning

confidence: 99%

The Validity of a Simplified Version of Baddeley's Three‐minute Reasoning Test

Hartley

Holt

1971

Educational Research

View full text Add to dashboard Cite

THIS paper describes an assessment of the validity and reliability of Baddeley's Three-Minute Reasoning Test following its adaptation by the authors for use with school children. The results indicated that there was a significant correlation between scores on the Three Minute Test and the AH4 (+0 . 70) and that correlations between performance and scores on the AH4 and performance and scores on the reasoning test were of similar size. A test-retest correlation, however, obtained over a seven-day interval, suggested that the reliability of the test was low (0 . 66).

show abstract

Short-form tests: A methodological review.

Cited by 98 publications

References 57 publications

Test Length and Decision Quality in Personnel Selection: When Is Short Too Short?

Test Length and Decision Quality in Personnel Selection: When Is Short Too Short?

Administration time estimates for Wechsler Intelligence Scale for Children‐IV subtests, composites, and short forms

The Validity of a Simplified Version of Baddeley's Three‐minute Reasoning Test

Contact Info

Product

Resources

About