The evaluation of professional competence in psychology, by licensing boards, the American Board of Professional Psychology, or other boards, has not taken full advantage of expertise in psychological measurement within the profession. This article explores the advantages and disadvantages of work samples and simulations for such purposes and compares them especially to paper-and-pencil tests. Multiple examples, drawn from various professions and from psychological research in personnel selection, illustrate the problems and possibilities. A comparison of work samples with simulations shows greater advantages for the latter type, but a combination of exercises in an assessment center model is finally recommended. The article concludes with an illustration of how examinations of psychological competence at the licensing and the specialty board levels could be improved by incorporating work samples and simulations.Various boards that grant credentials, such as the state licensing boards, the American Board of Professional Psychology (ABPP), and other nonstatutory boards, face the problem of evaluating professional competence to practice psychology. Unfortunately, such groups have yet to take advantage of the expertise in psychological measurement of those professionals they attempt to certify. The national licensing examination (Examination for Professional Practice in Psychology, or EPPP) is a knowledge-based paper-and-pencil test that measures psychological background rather than competence to practice psychology. Oral examinations or essays may be used by licensing boards to focus on practice, but they lack the standardization of the EPPP. The ABPP examination relies on work samples brought in by candidates but it also lacks standardization; reliability and validity remain unchecked. Other boards have experimented with measurement methods but have so far failed to establish their reliability and validity for the evaluation of competence. In evaluating itself, psychology has not stood up to its own standards.This article offers one step toward improving that record by looking at the accumulated evidence for and against the use of work samples and simulations for competency evaluation. Evidence and examples are drawn from two primary sources: the use of such tools for competency evaluation in various professions (often with the advice and consultation of psychologists) and the experience in personnel selection of many industrial/organizational psychologists.
History and RationaleHistorically, the personnel selection literature has been dominated by the use of paper-and-pencil tests. This trend was promulgated during the two world wars, when general intelligence tests and later multiple-aptitude batteries proved to be expedient ways to select military personnel (Anastasi, 1982). One deviation from this genre was the work directed by Henry Murray for the Office of Strategic Services (OSS), which used assessment center methods, including extensive simulations, to select espionage agents (OSS, 1948). But, as the work ...