A systematic review and taxonomy of tools for evaluating evidence-based medicine teaching in medical education

Abstract

Background: The importance of teaching the skills and practice of evidence-based medicine (EBM) for medical professionals has steadily grown in recent years. Alongside this growth is a need to evaluate the effectiveness of EBM curriculum as assessed by competency in the five 'A's': asking, acquiring, appraising, applying and assessing (impact and performance). EBM educators in medical education will benefit from a compendium of existing assessment tools for assessing EBM competencies in their settings. The purpose of this review is to provide a systematic review and taxonomy of validated tools that evaluate EBM teaching in medical education. Methods: We searched MEDLINE, EMBASE, Cochrane library, Educational Resources Information Centre (ERIC), Best Evidence Medical Education (BEME) databases and references of retrieved articles published between January 2005 and March 2019. We have presented the identified tools along with their psychometric properties including validity, reliability and relevance to the five domains of EBM practice and dimensions of EBM learning. We also assessed the quality of the tools to identify high quality tools as those supported by established interrater reliability (if applicable), objective (non-self-reported) outcome measures and achieved ≥ 3 types of established validity evidence. We have reported our study in accordance with the PRISMA guidelines. Results: We identified 1719 potentially relevant articles of which 63 full text articles were assessed for eligibility against inclusion and exclusion criteria. Twelve articles each with a unique and newly identified tool were included in the final analysis. Of the twelve tools, all of them assessed the third step of EBM practice (appraise) and four assessed just that one step. None of the twelve tools assessed the last step of EBM practice (assess). Of the seven domains of EBM learning, ten tools assessed knowledge gain, nine assessed skills and-one assessed attitude. None addressed reaction to EBM teaching, self-efficacy, behaviours or patient benefit. Of the twelve tools identified, six were high quality. We have also provided a taxonomy of tools using the CREATE framework, for EBM teachers in medical education. Conclusions: Six tools of reasonable validity are available for evaluating most steps of EBM and some domains of EBM learning. Further development and validation of tools that evaluate all the steps in EBM and all educational outcome domains are needed. Systematic review registration: PROSPERO CRD42018116203.

A Pragmatic Trial of Active Versus Passive Teaching for Clinically Integrating Evidence Based Medicine Teaching in an Undergraduate Medical School.

“…Further evidence supports the view that EBM teaching and learning strategies should focus on implementing multi-faceted, clinically integrated approaches with assessments of knowledge, skills and behaviour in the medium to long term using validated assessment tools (14). To this end, a taxonomy of validated assessment tools has been developed for medical educators to aid in the evaluation of their EBM teaching (15). Though there is an increasing focus on clinically integrating EBM teaching, there is paucity of evidence on the most e cient way to integrate EBM teaching into clinical rotations.…”

“…Early evaluation of our curriculum in the rst two years of MBChB has shown an increase in students' knowledge as assessed by their performance in the Fresno test, in addition to their self-rated EBM knowledge and self-reported attitudes about EBM (Kumaravel et al, 2020, article in press). The aim of this study is to describe our approach of integrating EBM into clinical placements in an undergraduate medical school and to evaluate the effectiveness of two different models of EBM teaching (active and passive) as assessed by students' performances in their formative and summative assessments.…”

A prospective study evaluating the integration of a multifaceted evidence-based medicine curriculum into early years in an undergraduate medical school

“…The validated Fresno test [16] was used to assess the EBM competence of students before and after EBM teaching in Phase I. The Fresno test has been identified as a high quality tool as supported by established interrater reliability, use of objective (non-self-reported) outcome measure(s) and has demonstrated multiple (≥3) types of established validity evidence (including evidence of discriminative validity) [4,15,21]. The Fresno test has established content validity; an inter rater reliability of 0.76-0.98 for individual items; Cronbach's alpha of 0.88; Item total correlation of 0.47-0.75; Item discrimination of 0.41-0.86 and construct validity-on the 212 point test, the novice mean was 95.6 and the expert mean was 147.5 (p < 0.001) [16].…”

Development and evaluation of a spiral model of assessing EBM competency using OSCEs in undergraduate medical education

“…An evidence-based approach is considered a core competency for clinicians and tremendous efforts have been made to embed EBM training in both undergraduate and postgraduate medical curricula [3,4]. Various options of teaching EBM in undergraduate medical curriculum have been explored, ranging from standalone courses to those integrated with clinical teaching [4][5][6]. Teaching EBM as a longitudinal theme across the medical curriculum has been shown to be effective in improving EBM knowledge, as demonstrated by students' performances in the validated Fresno and Berlin tests [7].…”