CanMEDS evaluation in Canadian postgraduate training programmes: tools used and programme director satisfaction

Chou, Sophia; Cole, Gary; McLaughlin, Kevin; Lockyer, Jocelyn

doi:10.1111/j.1365-2923.2008.03111.x

Cited by 102 publications

(114 citation statements)

References 15 publications

Supporting

Mentioning

105

Contrasting

Unclassified

Order By: Relevance

“…Concurrent validity testing with other tools, such as OSCEs and DECs, assessing clinical competence outlined by the CanMEDs roles is also warranted in EM and other training programs. 3,9 Although the use of an ITER may allow for the evaluation of the intrinsic domains such as the Health Advocate and Collaborator roles, which have been regarded as extraneous in EM compared to the Medical Expert, Communicator, and Professional roles, further exploration into the situational factors that affect clinical performance is also needed. From a measurement point of view, it is more feasible to measure scenarios and participants in isolation than to decipher their interactions according to Lurie, who states, ''Descriptions of 'competencies' should thus specify the range of examinees and situations to which they apply.…”

Section: Discussionmentioning

confidence: 99%

“…A survey of program directors showed concern about how the non-Medical Expert or intrinsic CanMEDS roles are evaluated in postgraduate medical education programs. 3 Although the use of multiple tools may be recommended when assessing the CanMEDS roles, the utility and feasibility of such a task within resident training curricula needs further investigation. As Lurie suggests, 4 the use of multiple and different assessment methods need to be based on the comparability of these measures as a function of whether the performance measured represents a stable characteristic of the examinee rather than unique elements derived from prescriptive testing situations.…”

mentioning

confidence: 99%

“…2,3,5 To date, the ITER (as a summative measure based on preceptors' observations over typical block rotation periods) has been recognized as the one tool that can best capture an assessment of all seven CanMEDS roles. 2,3 The ITER is completed by the clinical assessor based on a number of observed clinical performances over the course of a resident's training period (e.g., 8-week block rotation) and can be an efficient way to conduct a broad evaluation of domain and expected behaviour areas. It allows the evaluator to use multiple sources of information and provides a summative evaluation of the resident's clinical performance.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Validity and reliability of an in-training evaluation report to measure the CanMEDS roles in emergency medicine residents

Kassam¹,

Donnon²,

Rigby

2014

CJEM

View full text Add to dashboard Cite

Background: There is a question of whether a single assessment tool can assess the key competencies of residents as mandated by the Royal College of Physicians and Surgeons of Canada CanMEDS roles framework. Objective: The objective of the present study was to investigate the reliability and validity of an emergency medicine (EM) in-training evaluation report (ITER). Method: ITER data from 2009 to 2011 were combined for residents across the 5 years of the EM residency training program. An exploratory factor analysis with varimax rotation was used to explore the construct validity of the ITER. A total of 172 ITERs were completed on residents across their first to fifth year of training. Results: A combined, 24-item ITER yielded a five-factor solution measuring the CanMEDs role Medical Expert/ Scholar, Communicator/Collaborator, Professional, Health Advocate and Manager subscales. The factor solution accounted for 79% of the variance, and reliability coefficients (Cronbach alpha) ranged from a 5 0.90 to 0.95 for each subscale and a 5 0.97 overall. The combined, 24-item ITER used to assess residents' competencies in the EM residency program showed strong reliability and evidence of construct validity for assessment of the CanMEDS roles. Conclusion: Further research is needed to develop and test ITER items that will differentiate each CanMEDS role exclusively.

show abstract

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Validity and reliability of an in-training evaluation report to measure the CanMEDS roles in emergency medicine residents

Kassam¹,

Donnon²,

Rigby

2014

CJEM

View full text Add to dashboard Cite

show abstract

“…A survey of Canadian residency programs demonstrated an average of 1.75 assessment methods. 5 Of equal importance is the frequency and consistency of formative assessment, 6 its integration into the educational curriculum, and the "catalytic effect" 7 of assessment results and feedback on improving individual performance. The selection of assessment methods will be based at least in part on the availability of financial, faculty, and learning resources within a residency.…”

Section: Overview Of Assessment Methods Identifiedmentioning

confidence: 99%

Assessing Patient Care: Summary of the Breakout Group on Assessment of Observable Learner Performance

Takayesu

Kulstad

Wallenstein

et al. 2012

Academic Emergency Medicine

View full text Add to dashboard Cite

There is an established expectation that physicians in training demonstrate competence in all aspects of clinical care prior to entering professional practice. Multiple methods have been used to assess competence in patient care, including direct observation, simulation-based assessments, objective structured clinical examinations (OSCEs), global faculty evaluations, 360-degree evaluations, portfolios, self-reflection, clinical performance metrics, and procedure logs. A thorough assessment of competence in patient care requires a mixture of methods, taking into account each method's costs, benefits, and current level of evidence. At the 2012 Academic Emergency Medicine (AEM) consensus conference on educational research, one breakout group reviewed and discussed the evidence supporting various methods of assessing patient care and defined a research agenda for the continued development of specific assessment methods based on current best practices. In this article, the authors review each method's supporting reliability and validity evidence and make specific recommendations for future educational research.ACADEMIC EMERGENCY MEDICINE 2012; 19:1379-1389 by the Society for Academic Emergency Medicine I n 2001, the Accreditation Council for Graduate Medical Education (ACGME) introduced a timeline for the implementation of training and assessment in six core competencies that form the foundation of clinical competence. Introduced in 1996, the Canadian CanMEDS manager competency correlates to the AC-GME patient care competency, broadly defined as "the active engagement in decision-making in the operation of the healthcare system."1 The patient care competency for emergency medicine (EM) has been defined by a previous Academic Emergency Medicine (AEM) consensus conference, 2 now further elaborated on by the milestones in training, 3 as being able to efficiently gather and synthesize medical and diagnostic information, prioritize tasks, and implement management plans on multiple patients, as well as performing essential invasive procedures competently.There is an explicit expectation that physicians in training demonstrate competence in various aspects of clinical care prior to graduation and professional practice. 4 While this accountability falls squarely on the shoulders of residency training programs, it is mirrored by commensurate expectations of maintenance of competency during ongoing professional practice.The goals of the 2012 AEM consensus conference patient care working group were to describe the current state of evidence for assessment of competence in patient care and define a research agenda for the further development of specific assessment methods based on current best practices. METHODSA search was conducted using MEDLINE 1996-present using the key word search terms "assessment," "patient care," "competency," "competence," "assess*," "emergency," and "education" and limited to humans and English language [boolean search: ((assessment and patient care AND (competency or competence)) OR (assess* a...

show abstract

“…Residents may have limited experience dealing with these less common scenarios, and the CanMEDS OSCE provides a direct assessment method for these specific competencies. Interestingly, in a survey of residency program directors from across Canada, it was revealed that they were least satisfied with the assessment of the Health Advocate Role (Chou, et al, 2008). This further highlights the potential utility of the CanMEDS OSCE as examiners can control case content to specifically and directly evaluate Roles that may not necessarily occur on a regular basis during day-to-day practice.…”

Section: Discussionmentioning

confidence: 99%

Assessment of the Intrinsic CanMEDS Roles in Diagnostic Radiology Residents using an Objective Structured Clinical Assessment (OSCE)

Probyn¹,

Lang²,

Finlay³

et al. 2016

MedEdPublish

View full text Add to dashboard Cite

Purpose: Non-Medical Expert or Intrinsic CanMEDS Roles, as outlined by the Royal College of Physicians and Surgeons of Canada, can be difficult to evaluate in an objective and specialty-specific manner. This study investigates an Objective Structured Clinical Examination (OSCE) evaluation tool to assess these competencies for Diagnostic Radiology Residents. Methods:A five station CanMEDS OSCE was developed for postgraduate year 3 and 4 Residents to evaluate the Communicator, Collaborator, Manager, Health Advocate, Scholar and Professional Roles. Performance was assessed by postgraduate year 5 Residents using standardized scoring rubrics. CanMEDS OSCE scores were correlated with American College of Radiology (ACR) scores and Medical Expert OSCEs.Results: Seventy Residents in three separate cohorts (n=21, 26, 23) participated in the CanMEDS OSCE. Mean station scores were consistent across cohorts. In general, one-way ANOVAs showed no effect of postgraduate year on station scores. There were no significant correlations between CanMEDS OSCE scores and ACR exam scores or CanMEDS OSCE scores and Medical Expert OSCE scores, demonstrating divergent construct validity. In turn, this indicates construct validity for the CanMEDS OSCE by demonstrating that unique competencies are being measured. In contrast, there was a correlation between ACR exam scores and Medical Expert OSCE scores, confirming that these both assess the same construct.Conclusions: An OSCE can be a useful assessment tool to assess Intrinsic CanMEDS Roles in a specialty-specific manner. Correlational analyses indicate that unique competencies are being evaluated that are not captured by other traditional assessment means.

show abstract

CanMEDS evaluation in Canadian postgraduate training programmes: tools used and programme director satisfaction

Cited by 102 publications

References 15 publications

Validity and reliability of an in-training evaluation report to measure the CanMEDS roles in emergency medicine residents

Validity and reliability of an in-training evaluation report to measure the CanMEDS roles in emergency medicine residents

Assessing Patient Care: Summary of the Breakout Group on Assessment of Observable Learner Performance

Assessment of the Intrinsic CanMEDS Roles in Diagnostic Radiology Residents using an Objective Structured Clinical Assessment (OSCE)

Contact Info

Product

Resources

About