The relationship between learner engagement and teaching effectiveness: a novel assessment of student engagement in continuing medical education

Abstract

Background Continuing medical education (CME) often uses passive educational models including lectures. However, numerous studies have questioned the effectiveness of these less engaging educational strategies. Studies outside of CME suggest that engaged learning is associated with improved educational outcomes. However, measuring participants’ engagement can be challenging. We developed and determined the validity evidence for a novel instrument to assess learner engagement in CME. Methods We conducted a cross-sectional validation study at a large, didactic-style CME conference. Content validity evidence was established through review of literature and previously published engagement scales and conceptual frameworks on engagement, along with an iterative process involving experts in the field, to develop an eight-item Learner Engagement Instrument (LEI). Response process validity was established by vetting LEI items on item clarity and perceived meaning prior to implementation, as well as using a well-developed online platform with clear instructions. Internal structure validity evidence was based on factor analysis and calculating internal consistency reliability. Relations to other variables validity evidence was determined by examining associations between LEI and previously validated CME Teaching Effectiveness (CMETE) instrument scores. Following each presentation, all participants were invited to complete the LEI and the CMETE. Results 51 out of 206 participants completed the LEI and CMETE (response rate 25%) Correlations between the LEI and the CMETE overall scores were strong (r = 0.80). Internal consistency reliability for the LEI was excellent (Cronbach’s alpha = 0.96). To support validity to internal structure, a factor analysis was performed and revealed a two dimensional instrument consisting of internal and external engagement domains. The internal consistency reliabilities were 0.96 for the internal engagement domain and 0.95 for the external engagement domain. Conclusion Engagement, as measured by the LEI, is strongly related to teaching effectiveness. The LEI is supported by robust validity evidence including content, response process, internal structure, and relations to other variables. Given the relationship between learner engagement and teaching effectiveness, identifying more engaging and interactive methods for teaching in CME is recommended.

Perceptions and behaviors of learner engagement with virtual educational platforms

“…21 As learner engagement has been strongly associated with teaching effectiveness, it is key for educators to maximize engagement with the virtual platform. 22 This work has limitations. Our response rate of 13% is lower than desirable, and increases the risk of unit non-response bias i.e.…”

Active learning strategies in business education: using the law to build critical workforce skills

“…Moreover, the questionnaire also investigated the learners' level of engagement. Engagement is a complex phenomenon involving both physical and psychological Active learning strategies constructs (Stephenson et al, 2020). It has been defined as the amount of energy learners physically and psychologically invest to stay attentive, involved and motivated (Bernard, 2015;Astin, 1984).…”

“…It has been defined as the amount of energy learners physically and psychologically invest to stay attentive, involved and motivated (Bernard, 2015; Astin, 1984). Studies have demonstrated that more engaged learners are more satisfied and have better performance concerning course learning objectives (Stephenson et al , 2020; Bernard, 2015; Astin, 1984.) In this study, engagement was measured through the understanding of course content and the level of attention in class (Figures 4 and 5).…”

Emergency Medicine Virtual Conference Participants’ Engagement with Competing Activities

“…This transition creates questions about effective education, which depends on learner engagement for success. 1 – 3 Engagement serves as an important and measurable link between instruction and educational outcomes and is defined as focused attention on a specific task. 4 , 5 Accordingly, multiple direct-observation tools to assess learner engagement and behaviors exist for the classroom.…”