Using a theory of change for evaluation: has the FAIMER international faculty development program improved the field of health professions education?

Abstract

Program theories have not been extensively used in evaluating faculty development programs in medical education. Ample evidence shows that a well-formulated program theory plays a pivotal role in program implementation and evaluation. Program theory links activities and expected outcomes using a logical process showing how they lead to long-term goals. It also develops appropriate metrics or indicators for assessing if those outcomes and activities really occurred. In this study, FAIMER's theory of change was adopted as a framework for evaluation. Survey data from FAIMER Fellows was used to assess the effectiveness of FAIMER's faculty development program in meeting the goal of improving health professions education. We used structural equation modeling to examine the association among outcomes mapped out in our theory of change and their association with improving field of health professions education. The study results indicated that FAIMER's faculty development program appeared to have positively influenced advancement of multiple facets of health professions education as envisaged in our theory of change. Using a theoretical framework for evaluating a program helped us identify the specific areas of outcomes that need to be strengthened for program improvement as well as provided us with a data-driven evaluation framework to measure program progress.

FAIMER Global Faculty Development: A Sustainable Partnership Model to Advance Health Professions Education

“…Fellows complete a questionnaire on their project impact; 22 it is first administered when they start year 2 of the fellowship and then annually once they complete their fellowship (n = 1,223 fellows from 2001 to 2020, response rate 64%). This questionnaire, administered to all participants since 2015, 27 inquires about the status of their FAIMER projects and project-driven changes within their school, community, or region. The top 3 changes associated with projects include increased quality of education (999 [82%]), changes in institutional or departmental policies (569 [47%]), and increased relevance of education (546 [45%]).…”

Concluding Commentary: A Call for evidence in CPD and Lifelong Learning

“…The program helped the authors to identify specific areas to strengthen and thus improving faculty development programs and its measurement. ( Yuan S, Mukherjee S, Vyas R, Burdick W, 2019 )…”

Success stories: exploring the long-term impact of a global faculty development program for health professions educators

“…In another study, Baker et al (2010) highlighted the collaborative network structure in facilitating the participation of members who are geographically distributed. One of the principles of FAIMER FDPs is to build and sustain an online CoP for global health professions educators to foster networking, exchanging ideas and collaborations(22,35). During their inception, each FAIMER FDP had their own local online CoP through listservs.…”