In large-scale performance-based assessment of medical professionals' clinical competence, simulated patients (standardized patients-SPs) are used not only to simulate case problems but also to record on check-list the examinees' clinical performance during their encounter with the SPs. The purposes of this study were to determine the SPs' overall accuracy in recording check-list items, and whether their accuracy was affected by the various characteristics of the check-list, and by the different times during a day and the different days within the examination when the recordings were obtained. Results showed that the SPs' accuracy in recording check-list items was good to very good and was affected by the length of the check-list, as well as by the type and the clarity of the check-list item. It was further found that the SPs' accuracy was consistent and did not vary over the course of a one-day or a 15-day examination.

Doctor ratings of clerkship performance are often discounted as not accurately reflecting clinical competence. Such ratings are influenced by the following uncontrolled variables: case difficulty; differing rater focus and standards; lack of agreement on what constitutes acceptable performance; and collective patient care responsibility masks individual contributions. Standardized direct measures of clinical competence were developed to control these factors and allow direct comparisons of student performance. Students saw 18 patients representing frequently occurring and important patient problems. Student actions and decisions were recorded and subsequent responses to questions revealed knowledge of pathophysiology, basis for actions, use and interpretation of laboratory investigations, and management. Actions and responses were graded using a pre-set key. The examination covered 73% of designated clinical competencies. Examinations scores corresponded with independent measures of clinical competence. Reliability studies indicated that new cases can be substituted in subsequent years with confidence that scores will maintain similar meaning. Costs are +6.95 per student per case, which is modest considering the quality and quantity of information acquired. Methods described are practical for evaluation of clerks and residents and for licensing and specialty certification examinations.

By the end of 1990-91, the Southern Illinois University School of Medicine had had six years of experience with comprehensive, performance-based examinations of senior medical students' levels of clinical competence; this report assesses the psychometric aspects of the six examinations given during that period. The examinations were aimed at determining the students' readiness for postgraduate training. Compared with other clinical performance-based assessments that use standardized patients (SPs), these examinations had two important and unique features: (1) the examinations assessed a comprehensive range of clinical skills and reasoning; and (2) they approximated the challenges of real clinical practice wherein a practitioner's skills need to be orchestrated and prioritized in order to meet the challenges of the case encountered. Each year, the performance-based assessment given was an intensive clinical examination requiring each student to work up 13 to 18 SP problems over a three-day period. To administer an examination to an entire class of students took three weeks. Because all students after the first year of administration (1986) were required to pass these examinations, the fairness of test design and scoring and the setting of performance standards for the examinations became important issues for the faculty. The results, accumulated over six years and based on a total of 6,804 student-patient encounters involving 405 students, indicate that this kind of clinical performance-based examination can discriminate a wide range of students' clinical performances. The results provide evidence for the examinations' test security, content validity, construct validity, and reliability.

Except for the women examinee's higher performance in personal manner, the men and women examinees generally performed equally well with respect to interpersonal and communication skills, and they performed equally well regardless of the gender of the SP.

1994) Effect of using multiple standardized patients to rate interpersonal and communication skills on intercase reliability, Teaching