Evaluation of large-group lectures in medicine – development of the SETMED-L (Student Evaluation of Teaching in MEDical Lectures) questionnaire

Abstract

BackgroundThe seven categories of the Stanford Faculty Development Program (SFDP) represent a framework for planning and assessing medical teaching. Nevertheless, so far there is no specific evaluation tool for large-group lectures that is based on these categories. This paper reports the development and psychometric validation of a short German evaluation tool for large-group lectures in medical education (SETMED-L: ‘Student Evaluation of Teaching in MEDical Lectures’) based on the SFDP-categories.MethodsData were collected at two German medical schools. In Study 1, a full information factor analysis of the new 14-item questionnaire was performed. In Study 2, following cognitive debriefings and adjustments, a confirmatory factor analysis was performed. The model was tested for invariance across medical schools and student gender. Convergent validity was assessed by comparison with results of the FEVOR questionnaire.ResultsStudy 1 (n = 922) yielded a three-factor solution with one major (10 items) and two minor factors (2 items each). In Study 2 (n = 2740), this factor structure was confirmed. Scale reliability ranged between α = 0.71 and α = 0.88. Measurement invariance was given across student gender but not across medical schools. Convergent validity in the subsample tested (n = 246) yielded acceptable results.ConclusionThe SETMED-L showed satisfactory to very good psychometric characteristics. The main advantages are its short yet comprehensive form, the integration of SFDP-categories and its focus on medical education.Electronic supplementary materialThe online version of this article (doi:10.1186/s12909-017-0970-8) contains supplementary material, which is available to authorized users.

Presentation and evaluation of the teaching concept “ENHANCE” for basic sciences in medical education

“…Study 2: Comparative self-assessment (CSA) of ILOs [ 18 , 19 ] and the Student Evaluation of Teaching in MEDical Lectures (SETMED-L) questionnaire [ 20 ].…”

“…Teaching quality was assessed using a 14-item questionnaire that has been developed to specifically evaluate lectures in medical education [ 20 ]. The questionnaire consists of one major factor (“Core teaching skills”, ten items, e.g., “Session is well structured”) and two minor factors (“Student activation skills”, two items, e.g., “Teacher asks questions to check student learning outcome”; “Student workload”, two items, e.g., “Teaching pitched to the student level”).…”

“…In Study 2, differences in the teaching quality of the ISS courses were assessed using the Kruskal-Wallis test since skewed distributions, and ceiling effects were to be expected [20]. In the literature, different thresholds for ceiling effects are reported.…”

“…Furthermore, strong ceiling effects were detected in the present study. The problem of ceiling effects was also detected during the design and validation of the SETMED-L questionnaire [20], and the question came up whether ceiling effects could be eliminated if a sufficient range of the scale would be established to produce variability [29]. The fact that only one teacher was assessed in the present study made it difficult to discriminate whether the ceiling effect occurred because of the non-sufficient measurement design or because the teaching quality of this single teacher actually was excellent.…”

Qualitätssicherungssystem zur Bewertung eines HNO-Facharztrepetitoriums

“…Fragebögen sind eine häufig verwendete Methode, um effizient und routinemäßig Feedback von einer größeren Gruppe zu erhalten [13][14][15]. Ein Problem von Feedback ist jedoch häufig die Subjektivität [16,17].…”