Exploring the measurement properties of the osteopathy clinical teaching questionnaire using Rasch analysis

2020

BMC Med Educ

Self Cite

Background In a whole-of-system approach to evaluation of teaching across any degree, multiple sources of information can help develop an educators’ understanding of their teaching quality. In the health professions, student evaluations of clinical teaching are commonplace. However, self-evaluation of teaching is less common, and exploration of clinical educators’ self-efficacy even less so. The aim of the study was to evaluate how a clinical educator’s self-evaluation of teaching intersects with their self-efficacy, to ascertain if that matches student evaluation of their teaching. This information may assist in facilitating targeted professional development to improve teaching quality. Methods Clinical educators in the osteopathy program at Victoria University (VU) were invited to complete: a) self-evaluation version of the Osteopathy Clinical Teaching Questionnaire (OCTQ); and b) the Self-Efficacy in Clinical Teaching (SECT) questionnaire. Students in the VU program completed the OCTQ for each of the clinical educators they worked with during semester 2, 2017. Results Completed OCTQ and SECT were received from 37 clinical educators. These were matched with 308 student evaluations (mean of 6 student ratings per educator). Three possible educator cohorts were identified: a) high clinical eductor self-OCTQ with low student evaluation; b) low clinical educator self-evaluation and high student evaluations; and, c) no difference between self- and student evaulations. Clinical educators in the first cohort demonstrated significantly higher SECT subscale scores (effect size > 0.42) than their colleagues. Age, gender, teaching qualification, and years practicing or years as a clinical educator were not associated with clinical educator OCTQ scores or the SECT subscales. Conclusions Targeted professional development directed towards fostering self-efficacy may provide an avenue for engaging those clinical educators whose self-efficacy is low and/or those who did not receive high student evaluations. Given there is no gold standard measure of clinical teaching quality, educators should engage with multiple sources of feedback to benchmark their current performance level, and identify opportunities to improve. Student and self-evaluations using the OCTQ and evaluation of self-efficacy using the SECT, are useful tools for inclusion in a whole-of-system approach to evaluation of the clinical learning environment.

Section: Measures Studentsmentioning

confidence: 99%

“…Reliability estimates for the student OCTQ evaluations were calculated in R [46] using the the MBESS package [47]. Cronbach's alpha (α) and McDonald's omega total (ωt), and their respective confidence intervals were calculated consistent with Vaughan [42].…”

Section: Reliability Estimatesmentioning

confidence: 99%

Clinical educator self-efficacy, self-evaluation and its relationship with student evaluations of clinical teaching

2020

BMC Med Educ

Self Cite

“…Reliability estimates for the student OCTQ evaluations were calculated in R [46] using the the MBESS package [47]. Cronbach's alpha (a) and McDonald's omega total (wt), and their respective con dence intervals were calculated consistent with Vaughan [34].…”

Section: Reliability Estimatesmentioning

confidence: 99%

“…The Osteopathy Clinical Teaching Questionnaire (OCTQ) was developed to evaluate student perceptions of the quality of clinical teaching in their respective programs' in student-led, on-campus clinics [31], or university clinics [32]. Previous work provided evidence for the validity argument for the interpretation of scores derived from the OCTQ, including reliability (internal structure, test-retest, inter-rater), content validity, and structural validity [33][34][35]. Although the construct of 'quality of clinical teaching' has not been agreed on in the literature [36] -likely due to its context-speci c nature [37] -it broadly incorporates the interpersonal attributes, and teaching approaches utilised, by clinical educators [38], and is a term widely used in the literature [36,[39][40][41][42][43].…”

Section: Introductionmentioning

confidence: 99%

Clinical Educator Self-Efficacy, Self-Evaluation and Its Relationship With Student Evaluations of Clinical Teaching

2020

Preprint

Self Cite

Background In a whole-of-system approach to evaluation of teaching across any degree, multiple sources of information can help develop an educators’ understanding of their teaching quality. In the health professions, student evaluations of clinical teaching are commonplace however, self-evaluation of teaching is less common, and exploration of clinical educators’ self-efficacy even less so. The aim of the study was to evaluate how a clinical educator’s self-evaluation of teaching intersects with their self-efficacy, to ascertain if that matches student evaluation of their teaching. This information may assist in facilitating targeted professional development to improve teaching quality. Methods Clinical educators in the osteopathy program at Victoria University (VU) were invited to complete: a) self-evaluation version of the Osteopathy Clinical Teaching Questionnaire (OCTQ); and b) the Self-Efficacy in Clinical Teaching (SECT) questionnaire. Students in the VU program completed the OCTQ for each of the clinical educators they worked with during semester 2, 2017. Results Completed OCTQ and SECT were received from 37 clinical educators. These were matched with 308 student evaluations (mean of 6 student ratings per educator). Three possible educator cohorts were identified: a) high clinical eductor self-OCTQ with low student evaluation; b) low clinical educator self-evaluation and high student evaluations; and, c) no difference between self- and student evaulations. Clinical educators in the first cohort demonstrated significantly higher SECT subscale scores (effect size >0.42) than their colleagues. Age, gender, teaching qualification, and years practicing or years as a clinical educator were not associated with clinical educator OCTQ scores or the SECT subscales. Conclusions Targeted professional development directed towards fostering self-efficacy may provide an avenue for engaging those clinical educators whose self-efficacy is low and/or those who did not receive high student evaluations. Given there is no gold standard measure of clinical teaching quality, educators should engage with multiple sources of feedback to benchmark their current performance level, and identify opportunities to improve. Student and self-evaluations using the OCTQ and evaluation of self-efficacy using the SECT, are useful tools for inclusion in a whole-of-system approach to evaluation of the clinical learning environment.

Section: Reliability Estimatesmentioning

confidence: 99%

Clinical educator self-efficacy, self-evaluation and its relationship with student evaluations of clinical teaching

2020

Preprint

Self Cite

Background In a whole-of-system approach to evaluation of teaching across any degree, multiple sources of information can help develop an educators’ understanding of their teaching quality. In the health professions, student evaluations of clinical teaching are commonplace however, self-evaluation of teaching is less common, and exploration of clinical educators’ self-efficacy even less so. The aim of the study was to evaluate how a clinical educator’s self-evaluation of teaching intersects with their self-efficacy, to ascertain if that matches student evaluation of their teaching. This information may assist in facilitating targeted professional development to improve teaching quality. Methods Clinical educators in the osteopathy program at Victoria University (VU) were invited to complete: a) self-evaluation version of the Osteopathy Clinical Teaching Questionnaire (OCTQ); and b) the Self-Efficacy in Clinical Teaching (SECT) questionnaire. Students in the VU program completed the OCTQ for each of the clinical educators they worked with during semester 2, 2017. Results Completed OCTQ and SECT were received from 37 clinical educators. These were matched with 308 student evaluations (mean of 6 student ratings per educator). Three possible educator cohorts were identified: a) high clinical eductor self-OCTQ with low student evaluation; b) low clinical educator self-evaluation and high student evaluations; and, c) no difference between self- and student evaulations. Clinical educators in the first cohort demonstrated significantly higher SECT subscale scores (effect size >0.42) than their colleagues. Age, gender, teaching qualification, and years practicing or years as a clinical educator were not associated with clinical educator OCTQ scores or the SECT subscales. Conclusions Targeted professional development directed towards fostering self-efficacy may provide an avenue for engaging those clinical educators whose self-efficacy is low and/or those who did not receive high student evaluations. Given there is no gold standard measure of clinical teaching quality, educators should engage with multiple sources of feedback to benchmark their current performance level, and identify opportunities to improve. Student and self-evaluations using the OCTQ and evaluation of self-efficacy using the SECT, are useful tools for inclusion in a whole-of-system approach to evaluation of the clinical learning environment.