When less is more: validating a brief scale to rate interprofessional team competencies

Abstract

Background: There is a need for validated and easy-to-apply behavior-based tools for assessing interprofessional team competencies in clinical settings. The seven-item observer-based Modified McMaster-Ottawa scale was developed for the Team Objective Structured Clinical Encounter (TOSCE) to assess individual and team performance in interprofessional patient encounters.Objective: We aimed to improve scale usability for clinical settings by reducing item numbers while maintaining generalizability; and to explore the minimum number of observed cases required to achieve modest generalizability for giving feedback.Design: We administered a two-station TOSCE in April 2016 to 63 students split into 16 newly-formed teams, each consisting of four professions. The stations were of similar difficulty. We trained sixteen faculty to rate two teams each. We examined individual and team performance scores using generalizability (G) theory and principal component analysis (PCA).Results: The seven-item scale shows modest generalizability (.75) with individual scores. PCA revealed multicollinearity and singularity among scale items and we identified three potential items for removal. Reducing items for individual scores from seven to four (measuring Collaboration, Roles, Patient/Family-centeredness, and Conflict Management) changed scale generalizability from .75 to .73. Performance assessment with two cases is associated with reasonable generalizability (.73). Students in newly-formed interprofessional teams show a learning curve after one patient encounter. Team scores from a two-station TOSCE demonstrate low generalizability whether the scale consisted of four (.53) or seven items (.55).Conclusion: The four-item Modified McMaster-Ottawa scale for assessing individual performance in interprofessional teams retains the generalizability and validity of the seven-item scale. Observation of students in teams interacting with two different patients provides reasonably reliable ratings for giving feedback. The four-item scale has potential for assessing individual student skills and the impact of IPE curricula in clinical practice settings.Abbreviations: IPE: Interprofessional education; SP: Standardized patient; TOSCE: Team objective structured clinical encounter

Faculty Burnout in Pharmacy Education

“…These eight faculty members also evaluated the videos using the Interprofessional Collaborator Assessment Rubric (ICAR) and Modified McMaster-Ottawa (MMO) four-item scale to test the convergent and divergent validity of the PICT. 11,12 The ICAR and MMO were chosen for comparison with the PICT because of the overlap in the interprofessional criterion each measures as well as other aspects that made them less than ideal for use at our college. For instance, the MMO only has three competency levels, but other observer-based assessment tools used for skills-based assessment at the college had four competency levels.…”

Assessment tools in pre‐licensure interprofessional education: A systematic review, quality appraisal and narrative synthesis

“…Analysis of included articles demonstrated that the nursing profession was the most frequently represented (n = 27), 22‐48 followed by medicine (n = 20), 24,26,27,29,30,32,34‐36,38,40‐44,46‐50 and pharmacy (n = 14) 22,24‐26,28,29,31,41,43,44,50‐53 . A large number of studies were undertaken in the United States of America (n = 24), 22‐25,28‐33,36‐41,45,49‐52,54‐56 with a smaller number from Australia (n = 4), 34,43,57,58 Canada (n = 3) 53,59,60 and the United Kingdom (n = 2) 42,47 (see Appendix ).…”

Prediction equations of forced oscillation technique: the insidious role of collinearity

“…To avoid the detrimental effects of collinearity on a regression mode, several methods have been suggested. Redundant collinear or duplicate explanatory variables are often removed [ 36 , 37 ]. Collinear variables can also be combined into a single index.…”