Generalization of Cognitive and Noncognitive Validities across Personality‐based Job Families

Abstract

The positive relationship between complexity of work and the validity of general mental ability (GMA) measures across a variety of occupations is well supported by research and provides important practical and theoretical support for cognitive ability measures. However, there is currently no research demonstrating a systematic relationship between the size of the validities of any personality measure and the personality requirements of jobs, thus leaving open to question the predictive and construct validity of personality measures for applicant selection. We compared the validities of two biodata measures -one scored to measure social competence and one to measure GMA -across six job families that varied in social requirements. The validity of the GMA measure generalized at approximately the same level across the job families while the validity of the social competence measure decreased as social requirements diminished.

Open Peer Commentary and Author'S Response

“…Recent research has demonstrated that focusing on situational demands results in better understanding of crosssituational functionality of empirically keyed biodata. MacLane & Cucina (2015) found that criterion-keying biodata in jobs that required social competence was highly predictive of performance in other jobs that required interpersonal effectiveness but not as predictive for jobs less demanding. Personality psychologists may also find benefits from contextualizing questionnaires that ask about past biographical data.…”

Applying theory to the black box: A model for empirically scoring biodata

“…Within the biodata literature, published research has generally not focused on how situational job features affect biodata key generalization to other samples or different criteria. Although a few studies have explicitly addressed the impact of situational features on empirically keyed biodata validity (Carlson et al, ; Gandy, Dye, & MacLane, ; MacLane & Cucina, ; Rothstein et al, ), there are differences in how situational factors were conceptualized, and the lack of a unified framework have prevented a concrete understanding of how contextual factors affect the development of biodata keys.…”

“…Biodata studies that have examined the impacts of contextual factors on empirical biodata keying have taken somewhat diverse approaches. For example, MacLane and Cucina () keyed biodata items upon a specific behavioral criterion (social competence) and within a context where that behavioral domain was highly valued. They then showed that correlations between the scores on the biodata inventory and interpersonal performance were stronger in job families, where socially competent behavior was more important.…”

“…Individual differences are highly predictive in some scenarios but not in others (e.g., assertiveness) might be overlooked . MacLane and Cucina's study () offers exemplar use of job analysis to infer when the validity of biodata scores would be more transportable and is a starting point for the development of a more theoretical approach. Building off these studies, the next section introduces a model of empirical keying (Figure ) that explicates how item content, contextual demands, and criterion measures will affect the development of empirically derived biodata keys.…”

“…Although established models of validity exist, such as those describing the unitarian validity approach (Binning & Barrett, ), the proposed model is framed specifically in the context of empirical keying. This model builds off work that has demonstrated how contextual features of the calibration setting (i.e., the job features of the calibration sample) affect empirical biodata keys (Carlson, Scullen, Schmidt, Rothstein, & Erwin, ; MacLane & Cucina, ; Rothstein et al, ), and extends this by fully incorporating the interplay between item content, criterion choice, and contextual features of the calibration setting. These factors are used to make predictions regarding what empirically keyed biodata inventories are measuring (i.e., what knowledge, skills, abilities, and other characteristics, KSAOs) and how they will generalize to new contexts (i.e., criterion‐related validity).…”