Development and Validation of a Computerized Interpretation System for Personnel Tests

Abstract

A computerized system was developed for generating narrative interpretations of scores from a battery of personnel screening tests. The report structure and interpretive statement library were designed to capture the test expertise and interpretive strategies of a panel of testing experts. This was accomplished by enumerating the questions that the experts believed the battery could answer, developing answers to these questions, and devising rules for selecting the appropriate answers based on test‐battery scores. The accuracy, thoroughness, readability, and coherence of the computer‐generated reports were evaluated in comparison to reports generated by human experts for the same examinees. Results of the evaluation showed the computerized reports to be more accurate and thorough, as readable, and somewhat less coherent than interpretations generated by the typical human expert. The computerized system development and validation strategies described are useful for other applications in which numbers are interpreted in a narrative report format.

The Evolution of the Field of Human Resource Information Systems: Co-Evolution of Technology and HR Processes

“…In addition, the first paper published in MIS Quarterly was published by DeSanctis (1986) who investigated how organizations were using technology to support human resources. It is interesting that the first paper published in a major HR and psychology journal was also published in 1986 and that it is focused on using computers to interpret the results from selection tests (Vale, Keller, & Bentz, 1986).…”

Computer‐based Test Interpretations: Some Development and Application Issues

“…The test score estimates of the expert raters generally underestimated the obtained test scores. Vale and Keller (1984) concluded that the narratives of the automated interpretive system was comparable to the narratives of the manually derived interpretive system.…”

“…This meant that the expert interpreters were able to rate the output based on their understanding of the interpretive model. Vale and Keller (1984) conducted a validation study of an interpretive system that included measures in the following areas: mental ability, personality, values and interests. The four measures had been applied manually to screen individuals for an executive managerial training program.…”

Applied Computer Diagnostics — Theoretical Foundations and Future Perspectives

“…Although the general weaknesses in analysing test results cannot be denied, it has to be acknowledged that some programs go beyond scoring, assigning norms and graphically illustrating results: the interpretation of data (Booth 1986 Vale et al 1986) should also be considered in relation to the discussion about statistical vs. clinical judgement (Dawes 1971;Hanis 1987;Jager 1986,236ff .). Vale and Keller (1987) differentiate three types of approaches for interpreting data with computer assistance: -The term descriptive (program) systems is used to refer to systems that assign interpretative statements to test results.…”