The Assessment of Emergency Physicians by a Regulatory Authority

Abstract

Objectives: To determine whether it is possible to develop a feasible, valid, and reliable multisource feedback program (360 evaluation) for emergency physicians.Methods: Surveys with 16, 20, 30, and 31 items were developed to assess emergency physicians by 25 patients, eight coworkers, eight medical colleagues, and self, respectively, using five-point scales along with an ''unable to assess'' category. Items addressed key competencies related to communication skills, professionalism, collegiality, and self-management.Results: Data from 187 physicians who identified themselves as emergency physicians were available. The mean number of respondents per physician was 21.6 (SD AE 3.87) (93%) for patients, 7.6 (SD AE 0.89) (96%) for coworkers, and 7.7 (SD AE 0.61) (95%) for medical colleagues, suggesting it was a feasible tool. Only the patient survey had four items with ''unable to assess'' percentages R15%. The factor analysis indicated there were two factors on the patient questionnaire (communication/professionalism and patient education), two on the coworker survey (communication/collegiality and professionalism), and four on the medical colleague questionnaire (clinical performance, professionalism, self-management, and record management) that accounted for 80.0%, 62.5%, and 71.9% of the variance on the surveys, respectively. The factors were consistent with the intent of the instruments, providing empirical evidence of validity for the instruments. Reliability was established for the instruments (Cronbach's a > 0.94) and for each physician (generalizability coefficients were 0.68 for patients, 0.85 for coworkers, and 0.84 for medical colleagues). Conclusions:The psychometric examination of the data suggests that the instruments developed to assess emergency physicians were feasible and provide evidence for validity and reliability.

Multisource Feedback to Assess Surgical Practice: A Systematic Review

“…We excluded studies if they (1) were in nonsurgical specialties such as pediatrics, family medicine, obstetrics, and gynecology etc., (2) provided only general descriptions and information about MSF without empirical data, (3) reported only the process of MSF, and (4) only reported changes in performance after feedback.…”

Developing and Implementing a Multisource Feedback Tool to Assess Competencies of Emergency Medicine Residents in the United States

“…Target numbers for all questionnaires were based on prior work. 9 Data collected for the study were not provided to the resident's supervisors. All data were deidentified prior to analysis.…”

“…While there is validity evidence to support their use in practicing physicians, MSF instrument use in resident assessment has not been previously studied. 9 Furthermore, although the existing instruments were developed for episodic care, they have not been adapted specifically for the emergency department (ED) setting, which limits EM educators' ability to apply them to learner assessment.…”

Assessing Professionalism: Summary of the Working Group on Assessment of Observable Learner Performance

“…It is not professionalism-specific but aspects of professionalism make up a large part of the assessment. Lockyer et al 31 evaluated the tool in a population of practicing EPs. Their protocol required ratings from eight coworkers, eight colleagues, and 25 patients who were selected by the physician.…”