What else is happening? A more holistic view of programme evaluation

Brandl, Katharina; Mandel, Jess

doi:10.1111/medu.13524

Cited by 4 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An important limitation of the RIPLS is that it provides numerical results for psychological constructs that intrinsically lack quantitative structure [ 20 ]. Further, numerical rankings do not easily translate into a meaningful representation of the student experience [ 21 ] as the diversity of perceptions across individuals and groups are obscured by the calculation of a mean and standard deviation. This may be of particular importance for healthcare educators who aim to better understand students’ perceptions of IPE and adjust IPE curricula towards student needs and level of readiness.…”

Section: Introductionmentioning

confidence: 99%

Readiness for interprofessional learning among health science students: a cross-sectional Q-methodology and likert-based study

Oliveira,

Brewer-Deluce,

Akhtar-Danesh

et al. 2023

BMC Med Educ

View full text Add to dashboard Cite

Background Interprofessional education (IPE) prepares healthcare students for collaboration in clinical practice, but the effectiveness of this teaching method depends on students’ readiness for and perceptions of IPE. Evaluating students’ readiness for and perceptions of IPE is challenging, due to the lack of comprehensive measures. This study characterized the level of IPE readiness and perspectives across first-year undergraduate and graduate health science students using the readiness for interprofessional learning Likert Scale (RIPLS) and Q-methodologies. Methods This is a cross-sectional, online study. Students were randomized to answer the Likert-scale version of RIPLS (80%) or a matched Q-methodology survey (20%). An ANCOVA compared RIPLS scores between students from different program levels (graduate/undergraduate) and specialization (health professional and general programs). The Q-data was analysed using a by-person factor analysis. Results Three hundred and four (33% response rate) and 71 (30% response rate) students completed the Likert scale and the Q-methodology surveys, respectively. Students from graduate programs demonstrated high readiness for IPE (higher total RIPLS scores p < 0.001) in comparison to undergraduates. Three factors, associated with program specialization (p = 0.04), emerged from the Q-methodology analysis characterizing students learning priorities. Students in undergraduate general programs were focused on IPE relevance and benefits to “the clinical team”, students in graduate programs focused on “the patient”, and those in undergraduate health professional programs focused on themselves (“me”). Conclusions This novel mixed-methods approach combining traditional Likert-scales with Q-methodology elucidated not only associations between program and specialization with readiness (Likert) but also which components of IPE were valued the most (Q-methodology) and by whom.

show abstract

Section: Introductionmentioning

confidence: 99%

Readiness for interprofessional learning among health science students: a cross-sectional Q-methodology and likert-based study

Oliveira,

Brewer-Deluce,

Akhtar-Danesh

et al. 2023

BMC Med Educ

View full text Add to dashboard Cite

show abstract

“…Finally, while open‐ended survey responses appended to Likert scales may provide “a rich description of respondent reality,” they are challenging to analyze and are limited as some respondents may not answer questions completely (Jackson and Trochim, ), or opinions expressed may be inconsistent across a group. Therefore, while the need for course evaluations is established (Cronbach, ), those based on Likert scales may be underpowered for understanding the complexities of the student educational experience (Brandl and Mandel, ).…”

Section: Introductionmentioning

confidence: 99%

Beyond Average Information: How Q‐Methodology Enhances Course Evaluations in Anatomy

Brewer‐Deluce

Sharma

Akhtar–Danesh

et al. 2019

Anatomical Sciences Ed

View full text Add to dashboard Cite

Course evaluations can be used for curriculum improvement and have the potential to better the student learning experience. However, because most are based on Likert scales and open‐ended feedback, understanding diversity in student opinion and uncovering optimal options for course change and improvement are often difficult. Alternatively, Q‐methodology can be used to investigate patterns of thought within a group and may offer greater potential for course reform. This manuscript offers a tutorial‐based explanation of the three components of Q‐methodology studies (1) survey instrument development, (2) data collection, and (3) analysis and interpretation, then demonstrates, via case study, the use of Q‐methodology to evaluate a fourth‐year undergraduate pathoanatomy course. The goal of this article is to enable the reader to broadly apply Q‐methodology in other courses to gain insight and feedback beyond that offered by traditional Likert scale methods. As demonstrated through the pathoanatomy case study, Q‐methodology highlights groups (denoted by factors) of like‐minded students that share opinions, preferences, and values. Overall, Q‐methodology analyses support course instructors in identifying areas of course strength and improvement in an evidence‐based way. This alternative to traditional Likert scales represents a promising solution to ongoing course evaluation limitations.

show abstract

“…The evaluation of medical school courses requires a range of methods to gain a sufficiently comprehensive view of the program [ 1 , 2 ]. Most medical schools use quantitative methods in the form of closedend rating scales with 1 or 2 opportunities for open-ended comments [ 3 ].…”

mentioning

confidence: 99%

Benefits of focus group discussions beyond online surveys in course evaluations by medical students in the United States: a qualitative study

Brandl

Rabadia

Chang

et al. 2018

J Educ Eval Health Prof

Self Cite

View full text Add to dashboard Cite

In addition to online questionnaires, many medical schools use supplemental evaluation tools such as focus groups to evaluate their courses. Although some benefits of using focus groups in program evaluation have been described, it is unknown whether these inperson data collection methods provide sufficient additional information beyond online evaluations to justify them. In this study, we analyze recommendations gathered from student evaluation team (SET) focus group meetings and analyzed whether these items were captured in open-ended comments within the online evaluations. Our results indicate that online evaluations captured only 49% of the recommendations identified via SETs. Surveys to course directors identified that 74% of the recommendations exclusively identified via the SETs were implemented within their courses. Our results indicate that SET meetings provided information not easily captured in online evaluations and that these recommendations resulted in actual course changes.

show abstract

What else is happening? A more holistic view of programme evaluation

Abstract: Brandl et al. explores the benefits and challenges for educators seeking to implement a novel holistic evaluation approach proposed by Rojas et al.

Cited by 4 publications

References 11 publications

Readiness for interprofessional learning among health science students: a cross-sectional Q-methodology and likert-based study

Readiness for interprofessional learning among health science students: a cross-sectional Q-methodology and likert-based study

Beyond Average Information: How Q‐Methodology Enhances Course Evaluations in Anatomy

Benefits of focus group discussions beyond online surveys in course evaluations by medical students in the United States: a qualitative study

Contact Info

Product

Resources

About