From the Work One Knows the Worker: A Systematic Review of the Challenges, Solutions, and Steps to Creating Synthetic Validity

Steel, Piers; Huffcutt, Allen I.; Kammeyer‐Mueller, John D.

doi:10.1111/j.1468-2389.2006.00331.x

Cited by 27 publications

(48 citation statements)

References 123 publications

(142 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite this lack of support, there has been a recent resurgence of interest in synthetic validation techniques (Hoffman, Rashkovsky, & D’Egidio, 2007; Jeanneret & Strong, 2003; Johnson, 2007; Scherbaum, 2005; Steel, Huffcutt, & Kammeyer‐Mueller, 2006). Based on our collective experience in researching and applying synthetic validation techniques, we maintain that synthetic validation is a practically useful methodology that has great potential to advance the science and practice of industrial and organizational (I‐O) psychology.…”

mentioning

confidence: 99%

Validation Is Like Motor Oil: Synthetic Is Better

Johnson¹,

Steel

Scherbaum

et al. 2010

Ind. organ. psychol.

View full text Add to dashboard Cite

Although synthetic validation has long been suggested as a practical and defensible approach to establishing validity evidence, synthetic validation techniques are infrequently used and not well understood by the practitioners and researchers they could most benefit. Therefore, we describe the assumptions, origins, and methods for establishing validity evidence of the two primary types of synthetic validation techniques: (a) job component validity and (b) job requirements matrix. We then present the case for synthetic validation as the best approach for many situations and address the potential limitations of synthetic validation. We conclude by proposing the development of a comprehensive database to build prediction equations for use in synthetic validation of jobs across the U.S. economy and reviewing potential obstacles to the creation of such a database. We maintain that synthetic validation is a practically useful methodology that has great potential to advance the science and practice of industrial and organizational psychology.

show abstract

mentioning

confidence: 99%

Validation Is Like Motor Oil: Synthetic Is Better

Johnson¹,

Steel

Scherbaum

et al. 2010

Ind. organ. psychol.

View full text Add to dashboard Cite

show abstract

“…Coupled with the recent resurgence in research and writing on job component validity and synthetic validation (Scherbaum, 2005;Steel, et al, 2006;Hoffman, et al, 2007;Johnson, 2007;Steel & Kammeyer-Mueller, 2009), we believe that job component validity deserves wider application in practice.…”

Section: Discussionmentioning

confidence: 85%

A Test of a Proposed Method for Estimating Validity of a Multivariate Composite Predictor: Extending the Job Component Validity Model

Hoffman

Morris²,

Luck

2009

Psychol Rep

View full text Add to dashboard Cite

In this study, a proposed extension to the job component validity model from the Position Analysis Questionnaire was tested. Job component validity, a form of synthetic validation, allows researchers to select useful predictors and to estimate the criterion-related validity of tests based on conducting a job analysis which includes the Position Analysis Questionnaire. Morris and colleagues described a method for estimating the multiple correlation of a test battery assembled via job component validity estimates. In the current study, job component validity estimates, derived from the multiple correlation procedure proposed by Morris, et al., were compared to unit-weighted validity estimates obtained in a criterion-related validity study of six job progressions. The multivariate job component validity estimates were comparable to unit-weighted validity coefficients obtained using supervisory ratings as criteria. Multivariate job component validity estimates were conservative compared to corrected unit-weighted validity coefficients.

show abstract

“…This method requires you to describe jobs in terms of a relatively universal taxonomy of components, establish the validity of different classes of tests for each component, then build a validity estimate for particular tests or test batteries on the basis of what you know about the validities of the tests for each components and the intercorrelations of the tests and the components. This method has clear conceptual appeal, but despite 50 years of research on synthetic validity, applications of synthetic validity have not proved particularly successful (Steel et al, 2006). Advances in meta-analysis have made synthetic validation more feasible (Steel et al, 2006), but there is no sign that empirical validity studies will be replaced any time soon by synthetic estimates.…”

Section: Synthetic Validitymentioning

confidence: 98%

“…This method has clear conceptual appeal, but despite 50 years of research on synthetic validity, applications of synthetic validity have not proved particularly successful (Steel et al, 2006). Advances in meta-analysis have made synthetic validation more feasible (Steel et al, 2006), but there is no sign that empirical validity studies will be replaced any time soon by synthetic estimates.…”

Section: Synthetic Validitymentioning

confidence: 98%

9 Validity, Validation and Values

Murphy

2009

The Academy of Management Annals

View full text Add to dashboard Cite

Tests and assessments are used in organizations for a wide range of purposes, and it is the uses of tests, not the tests themselves, that are validated. As a result, the critical question is often not "Is this test valid?", but rather "Valid for what?". Tests normally have multiple uses and purposes in organizations, which may be defined and understood differently by different stakeholders, and tests might have as many validities as they have uses. The strengths and weaknesses of existing validation strategies are examined and compared in the light of the ways tests are used in organizations. Content validation often seems unconnected with the ways tests are used and interpreted in organizations, and is not always useful a strategy for validating tests. Criterionoriented validation methods (including sophisiticated variants, such as the validity generalization model) are often deficient because they apply a univariate strategy for evaluating what is clearly a multivariate phenomenon-i.e., use of test scores to make high-stakes decisions in organization. Multivariate models of validation provide an opportunity to integrate qualitatively different criteria (e.g., efficiency and equity) in evaluating the validity of a test as it is used in an organization. *

show abstract

From the Work One Knows the Worker: A Systematic Review of the Challenges, Solutions, and Steps to Creating Synthetic Validity

Cited by 27 publications

References 123 publications

Validation Is Like Motor Oil: Synthetic Is Better

Validation Is Like Motor Oil: Synthetic Is Better

A Test of a Proposed Method for Estimating Validity of a Multivariate Composite Predictor: Extending the Job Component Validity Model

9 Validity, Validation and Values

Contact Info

Product

Resources

About