Recent research on multiple choice questions has identified deficiencies of inadequate content-equivalence and item-writer bias. Systematic methods of writing multiple choice questions are being advocated as effective responses. This article describes preliminary development of a new item-writing method. Details of the procedure, called item modelling, are provided.

Many health professions include written examinations among their licensing procedures, and constructing these examinations poses special difficulties. For physician licensure the central dilemma is manifested by the longstanding tradition of undifferentiated licensure contrasting with the strong specialty orientation of contemporary physician training. This article details the authors' response to this problem and describes the resulting design of a new physician licensure examination. Using a combination of empirical data and expert judgment, descriptions of selected clinical encounters have been assembled as a practice model of a physician licensed for the delivery of general health care of patients. Application of an explicitly situational framework to the design of a physician licensure examination is unusual, and the approach is advocated for use with other health professions.

The introduction of a clinical skills examination (CSE) to Step 2 of the U.S. Medical Licensing Examination (USMLE) has focused attention on the design and delivery of large-scale standardized tests of clinical skills and raised the question of the appropriateness of evaluation of these competencies across the span of a physician's career. This initiative coincides with growing pressure to periodically assess the continued competence of physicians in practice. The USMLE CSE is designed to certify that candidates have the basic clinical skills required for the safe and effective practice of medicine in the supervised environment of postgraduate training. These include history taking, physical examination, effective communication with patients and other members of the health care team, and clear and accurate documentation of diagnostic impressions and plans for further assessment. The USMLE CSE does not assess procedural skills. As physicians progress through training and enter practice, both knowledge base and requisite technical skills become more diverse. A variety of indirect and direct measures are available for evaluating physicians, but, at present, no single method permits high-stake inferences about clinical skills. Systematic and standardized assessments make a contribution to comprehensive evaluations, but they retain an element of assessing capacity rather than authentic performance in practice. Much work is needed to identify the optimal combination of methods to be employed in support of programs to ensure maintenance of competence of practicing physicians.

Although the unitary view of test validity has gained support recently, it has real limitations where professional licensing examinations are concerned. A strategy for validation of professional licensure tests requires modifying conventional approaches in three ways. First, a theory of professions must be incorporated into the test development process so as to acknowledge the social character of professions. Second, the importance of test design in the validation of licensing tests should be enhanced. Third, the concept of construct validation must be expanded to accommodate the special features of content that inhere to professional licensure testing. Methods for accomplishing these three things are described and the implications of these and other views discussed.

This article reviews the current status ofSocial Judgment Theory as it applies to work on medical reasoning. Following an overview of the primary features of Social Judgment Theory and its associated analytic techniques, a review of pertinent research applications in medicine is provided. The review concludes with a discussion of two intellectual problem areas associated with the theory.