Synthetic Validity In A Small Company: A Demonstration

Abstract

In BriefTRADITIONAL methods of empirically validating tests for employee selection require large numbers of employees all doing essentially the same work. The small business organization is not able to meet this requirement; it is typically faced, therefore, with the options of not testing or of using clinical appraisals. An alternative has been proposed under the term "synthetic validity" in which empirical validation data can be used to infer the validity of a battery of tests even for situations in which the N is too small to permit conventional validation. This report describes research in a small company in which nearly every employee has a job that is appreciably different from the jobs of other employees. Despite the diversity, however, there is some overlapping of job descriptions. Seven elements of work proficiency which appeared in many jobs in the company were identified. Ratings for each employee to whom an element applied were obtained, along with a rating on over-all effectiveness. A battery of tests was validated against this group of criterion ratings, taken one at a time.The setting provided the opportunity for a demonstration of the concept of synthetic validity. Because of small N's, the results of the study cannot be taken as conclusive evidence of the superiority of synthetic validity over more traditional methods, but it can be taken as (1) a demonstration of the concept and (2) a model for the evaluation of the synthetic validity approach.

Validation Is Like Motor Oil: Synthetic Is Better

“…According to Trattner (1982), this definition of construct validity means that a selection procedure can be used when work behaviors (i.e., job components) are important in any occupation within a class of occupations, as long as there is criterion‐related validity evidence linking the procedure to the work behaviors for incumbents in that class. Trattner concluded that the synthetic validation approaches of Primoff (1959) and Guion (1965) were consistent with this interpretation of the Guidelines . The Johnson and Carter (in press) approach also appears to meet these requirements.…”

“…Guion (1965) developed an early attempt to demonstrate the application of the job requirements matrix approach to synthetic validation. In conducting a job analysis of the various jobs in a small organization, he found that the jobs in this organization contained differing combinations of seven job components.…”

From the Work One Knows the Worker: A Systematic Review of the Challenges, Solutions, and Steps to Creating Synthetic Validity

“…If this column can be effectively predicted, we have a working selection system. Although this capability is impressive, synthetic validity is unremarkable from a scientific perspective (Guion, 1965). It depends merely on establishing predictable relationships (e.g., between tasks and abilities), a core feature of most fields.…”

Determining the Factors Influential in the Validation of Computer-based Problem Solving Systems

“…Job Performance Hollenbeck & Whitener, 1988 Missile Systems (Simulation) Kheir & Holmes, 1978 Primofl', 1959 Personnel Guion, 1965 Job Analysis McCornUck et al, 1979 Baughman, 1980 other models re both structure and output Decision Analysis Cazalet, 1980 Regional Blood Supply Planning Cumming, 1976 Management EiloD,1974 Decision Making (Management) Henderson & Nutt, 1980 Global Modeling Richardson, 1978 SENSITIVITY Test for stability of the model: compare Medical (MYCIN) Analysis the model "against itself" for sensitivity Buchanan & to slight changes in data, weights, Shortliffe, 1984 knowledge, etc, World Iron & Steel Economy Hashimoto, 1983 Wilderness Recreation Shechter & Lucas, 1980 Water Ecosystems Rechnagel & Benndorf, 1982 …”