Estimation of employment test validities by expert judgment.

Schmidt, Frank L.; Hunter, John E.; Croll, Paul R.; McKenzie, Robert C.

doi:10.1037/0021-9010.68.4.590

Cited by 26 publications

(38 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Empirical correlations between predictors and job components require large sample sizes and performance ratings, so subject matter expert judgments may be collected instead (Scherbaum, 2005). For example, test experts or I‐O psychologists could estimate the validity coefficients between predictors and job components (Schmidt, Hunter, Croll, & McKenzie, 1983). Research suggests that expert judgments provide an accurate estimate of these relationships, at least for cognitive ability tests and with experienced judges (Hirsh, Schmidt, & Hunter, 1986; Schmidt et al, 1983).…”

Section: Description Of Synthetic Validationmentioning

confidence: 99%

Validation Is Like Motor Oil: Synthetic Is Better

Johnson¹,

Steel

Scherbaum

et al. 2010

Ind. organ. psychol.

View full text Add to dashboard Cite

Although synthetic validation has long been suggested as a practical and defensible approach to establishing validity evidence, synthetic validation techniques are infrequently used and not well understood by the practitioners and researchers they could most benefit. Therefore, we describe the assumptions, origins, and methods for establishing validity evidence of the two primary types of synthetic validation techniques: (a) job component validity and (b) job requirements matrix. We then present the case for synthetic validation as the best approach for many situations and address the potential limitations of synthetic validation. We conclude by proposing the development of a comprehensive database to build prediction equations for use in synthetic validation of jobs across the U.S. economy and reviewing potential obstacles to the creation of such a database. We maintain that synthetic validation is a practically useful methodology that has great potential to advance the science and practice of industrial and organizational psychology.

show abstract

Section: Description Of Synthetic Validationmentioning

confidence: 99%

Validation Is Like Motor Oil: Synthetic Is Better

Johnson¹,

Steel

Scherbaum

et al. 2010

Ind. organ. psychol.

View full text Add to dashboard Cite

show abstract

“…The present study examined whether less experienced judges could also produce accurate estimates. Twenty-eight recent Ph.D.'s in 1/0 Psychology estimated observed validities for the same 54 job-test combinations used by Schmidt et al (1983). The estimates of these judges contained about twice as much random error as the experts' estimates.…”

mentioning

confidence: 91%

“…

…”

mentioning

confidence: 99%

Estimation of Employment Validities by Less Experienced Judges

Hirsh

Schmidt

Hunter

1986

Personnel Psychology

Self Cite

View full text Add to dashboard Cite

In a previous study, Schmidt, Hunter, Croll and McKenzie (1983) demonstrated that estimates of the validity of cognitive tests made by highly trained and experienced judges are more accurate than empirical estimates obtained from small‐sample validity studies. The present study examined whether less experienced judges could also produce accurate estimates. Twenty‐eight recent Ph.D.'s in I/O Psychology estimated observed validities for the same 54 job‐test combinations used by Schmidt et al. (1983). The estimates of these judges contained about twice as much random error as the experts' estimates. Systematic error of the less experienced judges was also greater than that of the experts (.0732 vs .019). The systematic errors of the two sets of judges were in opposite directions: less experienced judges overestimated validities, on average, while experts underestimated them. The results show that the estimates of less experienced judges contain less information than those of experts, but also that averages of estimates of several less experienced judges are as accurate as those obtained from small‐sample empirical studies.

show abstract

“…This approach was suggested originally by the apparent success of judgments regarding the tasks reported by Krause (Krause & Woldstad, 1983;Smith et al, 1983). Recent reports of the success of expert estimations of test validity provide support for this approach (Schmidt, Hunter, Croll, & McKenzie, 1983). Estimates of computer-original validity have approached their theoretical, reliability set, limits for several tests on the APT System.…”

Section: Discussionmentioning

confidence: 99%

Automated Portable Test (APT) System: Overview and prospects

Bittner¹,

Smith

Kennedy

et al. 1985

Behavior Research Methods, Instruments, & Computers

View full text Add to dashboard Cite

The Automated Portable Test (APT) System is a notebook-sized, computer-based, humanperformance and subjective-status assessment system. It is now being used in a wide range of environmental studies (e.g., simulator aftereffects, flight tests, drug effects, and hypoxia). Three questionnaires and 15 performance tests have been implemented, and the adaptation of 30 more tests is underway or is planned. The APT System is easily transportable, is inexpensive, and has the breadth of expansion options required for field and laboratory applications. The APT System is a powerful and expandable tool for human assessment in remote and unusual environments.The Automated Portable Test (APT) System has been developed as a tool for the assessment of human performance and subjective status. At present, it is being used in investigations of the effects of flight-simulator exposure on pilots and hypoxia effects on soldiers and in a variety of university studies. In the simulator study, for example, pre-and posttests are being administered in an effort to assess the magnitude and duration of simulator aftereffects on subjective and performance responses. Development of the APT System is based upon the concepts and empirical findings of the Performance Evaluation Tests for Environmental Research (PETER) Program

show abstract

Estimation of employment test validities by expert judgment.

Cited by 26 publications

References 7 publications

Validation Is Like Motor Oil: Synthetic Is Better

Validation Is Like Motor Oil: Synthetic Is Better

Estimation of Employment Validities by Less Experienced Judges

Automated Portable Test (APT) System: Overview and prospects

Contact Info

Product

Resources

About