Applying theory to the black box: A model for empirically scoring biodata

Speer, Andrew B.; Siver, Sydney R.; Christiansen, Neil Douglas

doi:10.1111/ijsa.12271

Cited by 9 publications

(12 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Biodata, defined as information about workers' previous experiences, behaviours, and feelings in specific situations (Stokes, 1999) have been used in personnel selection for over one hundred years (Becton et al 2009;Dean, 2013;Speer et al 2020). Biodata information contributes to the selection process by screening out applicants in terms of fit to organizational demands and job description and specification.…”

Section: Palabras Clave: Biodata Justicia Organizacional Selección Dementioning

confidence: 99%

Developing Biodata for Public Manager Selection Purposes: A Comparison between Fuzzy Logic and Traditional Methods

Izquierdo

Ramos-Villagrasa

Lubiano

2020

Revista De Psicología Del Trabajo Y De Las Organizaciones

View full text Add to dashboard Cite

Section: Palabras Clave: Biodata Justicia Organizacional Selección Dementioning

confidence: 99%

Developing Biodata for Public Manager Selection Purposes: A Comparison between Fuzzy Logic and Traditional Methods

Izquierdo

Ramos-Villagrasa

Lubiano

2020

Revista De Psicología Del Trabajo Y De Las Organizaciones

View full text Add to dashboard Cite

“…This makes sense, as empirical methods are explicitly designed to maximize prediction. The only instances in which empirical keys would be expected to result in lower validity would be if calibration sample sizes are small, or if the calibration sample differs greatly from the context in which operational biodata scales are to be used (Speer et al, 2020). The findings from this study should provide closure to the debate as to which method of biodata scoring will result in the highest criterion-related validity when predicting overall performance ratings.…”

Section: Discussionmentioning

confidence: 99%

“…Although empirical keying will score biodata inventories in a purely empirical manner, best practice still incorporates the development of questions based on job analysis or explicit domain sampling (Fine & Cronshaw, 1994; Russell, 1994). Thus, empirical keying does not preclude the presence of theory when developing biodata inventories, and contextual considerations and choice of criteria can further be incorporated to apply theoretical justification to the practice of empirical keying (Speer et al, 2020).…”

Section: What Is Biodata?mentioning

confidence: 99%

“…Rational scales are more interpretable and easier to communicate. Interpretability is relevant in many contexts, though there are ways that researchers can establish construct validity of empirically derived scales (Speer et al, 2020). Second, past comparisons between rational and empirical methods have sometimes suggested there is no difference in criterion-related validity between these scoring methods (e.g., Hough & Paullin, 1994; Jackson, 1975; Schoenfeldt, 1999).…”

Section: What Is Biodata?mentioning

confidence: 99%

See 1 more Smart Citation

Meta-analysis of biodata in employment settings: Providing clarity to criterion and construct-related validity estimates.

Tenbrink¹,

Speer²,

Sendra³

et al. 2022

Journal of Applied Psychology

Self Cite

View full text Add to dashboard Cite

Although biodata inventories have long been used to hire job applicants, there are limitations to current biodata knowledge and little in the way of contemporary biodata meta-analytic reviews. This study establishes a precise understanding of biodata validity by conducting an updated meta-analysis that differentiates biodata validity in terms of two important defining features: construct domain and scoring method (rational, hybrid, empirical). Evidence was established in terms of criterion-related validity with job performance and additional work outcomes, as well as convergent validity with common external hiring measures. In total, 180 independent samples of criterion correlations were examined, and 63 samples were analyzed containing correlations with convergent measures. Findings across the meta-analyses revealed that biodata inventories are one of the most predictive assessment methods available, but that the relationship with work outcomes differs by construct domain and scoring method. Empirically scored overall composite scales had superior criterion-related validity (ρ = .44) to rationally scored composite scales (ρ = .24). Scales developed to measure conscientiousness and leadership were generally the most predictive of the job performance of the narrow construct domains, and particularly when empirically keyed. However, when biodata scores were correlated with theoretically aligned performance ratings, rational scoring resulted in similar validity coefficients as empirical scoring. Finally, biodata scales exhibited expected patterns of correlations with external measures and were only modestly correlated with cognitive ability and five-factor model personality scores. Taken together, biodata inventories are highly predictive assessment methods and are likely to provide unique variance over other common predictors.

show abstract

“…If so, scores from a brief pretest of physical strength, such as hand-grip strength and a 1-minute push-up test, given at the time of application, should improve FIML estimates. More broadly, inclusion of keyed biodata (e.g., Cucina, Caputo, Thibodeaux, & MacLane, 2012;Speer, Silver, & Christiansen, 2020) and brief assessments of traits such as C and ES in application materials, should contribute to optimizing FIML estimates and provide "bonus" validity information for the biodata and personality scales involved. FIML and data imputation methods usually benefit from pretest data collection planning.…”

Section: Augmenting Pearson Rr Corrections With Missing Data Methodsmentioning

confidence: 99%

Bias in Observed Validity Estimates When Using Multiple Valid Predictors

Henderson

2021

Human Performance

View full text Add to dashboard Cite

Simulated data, validity reports and a firefighter predictive validation study are used to examine validity bias created by three common selection problems-range restriction, applicant and incumbent attrition, and nonlinearity created by compression of high selection test scores. Top 20% selection samples drawn from an applicant pool with known validity coefficients demonstrate that the sample validity estimates of the three predictors are differentially biased in both magnitude and direction, depending on the selection strategy used. Concurrent validity designs generally favor novel predictors. Corrections for direct range restriction across situations were mostly ineffectual. With proper scaling, corrections for indirect range restriction are accurate, but cross-variable biasing effects can occur when score distributions of the individual predictors differ. Many of the biases found in the simulation results are demonstrated in a firefighter predictive validation study where variations of Pearson-Thorndike range corrected validities and a full information maximum likelihood (FIML), approaches are all compared as validity assessments. With normalized predictors, both Pearson and FIML methods show that a test of general mental ability and physically demanding job tasks predicted firefighter performance throughout the 30-year study, with no evidence of interactions or a leveling of performance at high test scores.Pearson (1903) described the role of selection in altering means, variances, and correlations of variables in the selected sample, relative to the population from which the sample was drawn. He also demonstrated that altered sample variance effects differ when selection is explicit on a variable or operates indirectly on that variable. Pearson's conclusion that ". . . correlation can be created or destroyed or reversed by selection" (p. 29) was ignored by psychologists until Thorndike (1947) found that Pearson's equations explained the low predictive validity coefficients obtained in a highly selected sample of pilot applicants in WWII. Thorndike (1949) introduced the term range restriction (RR) and Pearson's equations for correcting bias in observed sample correlations, reframing Pearson's natural selection variance restriction orientation into bias correction relevant to personnel selection. Thorndike (1949) asserted:If any intelligent use be made of validity statistics from a restricted group, some statistical correction procedures are necessary to estimate what validity coefficients would have been obtained if it had been possible to obtain test and criterion data from a representative sample of all those to whom the selection devices were applied. (p. 171-172).He also concluded that indirect selection was the most common and important in personnel selection. Thorndike's treatment of range restriction and the necessity to use applicant pool referenced, not selected sample, validity coefficients, were widely accepted in textbooks (e.g., Guilford, 1956).

show abstract

Applying theory to the black box: A model for empirically scoring biodata

Cited by 9 publications

References 61 publications

Developing Biodata for Public Manager Selection Purposes: A Comparison between Fuzzy Logic and Traditional Methods

Developing Biodata for Public Manager Selection Purposes: A Comparison between Fuzzy Logic and Traditional Methods

Meta-analysis of biodata in employment settings: Providing clarity to criterion and construct-related validity estimates.

Bias in Observed Validity Estimates When Using Multiple Valid Predictors

Contact Info

Product

Resources

About