Assessing psychologists in practice: Lessons from the health professions using multisource feedback.

Abstract

The aim of the present study was to conduct a systematic literature review of multisource feedback (MSF) instruments and to summarize the evidence of feasibility, reliability, generalizability, validity, and other psychometric characteristics of the instruments. Accordingly, we conducted a systematic literature review for English-language studies published from 1975 to 2012 using the following databases: MEDLINE, EMBASE, CINAHL, PubMed, and PsycINFO. The following terms were used in the search: multisource feedback, 360-degree evaluation, and assessment of medical professionalism. Forty-eight studies conducted in Canada, the United States, the United Kingdom, Netherlands, China, and elsewhere met the inclusion criteria. The results indicate that MSF has adequate evidence of validity, reliability, and feasibility for providing health practitioners with quality improvement data (both formative and summative assessment) as part of an overall strategy of maintaining competence and certification. Professional psychology has not adopted MSF as a systematic competence-based method for evaluating, maintaining, and assuring competent practice of psychology and instead relies on self-assessment as the primary quality assurance approach for its public accountability. We make recommendations to adopt an MSF system of competence-based assessment of practicing psychologists by regulatory and licensing authorities in Canada and the United States.

Multisource feedback to graduate nurses: a multimethod study

“…Obtaining appraisals from people in multiple roles increases the perceived value of feedback (Overeem et al 2010) and multiple observations give the feedback strength (Atwater et al 2002, Brinkman et al 2007, Castanelli & Kitto 2011. Despite the benefits, CNEs in this study experienced difficulty engaging raters, an issue that has been identified as a common barrier when implementing the MSF process (Andrews et al 2013).…”

“…Despite the benefits, CNEs in this study experienced difficulty engaging raters, an issue that has been identified as a common barrier when implementing the MSF process (Andrews et al . ).…”

Assessing competence in sport psychology: An action research account

“…We discussed the needs, challenges, and available methods for assessment with stakeholders (such as students, assessors, and supervisors). In addition, we conducted a study of literature on competency assessment in sport psychology (e.g., Fletcher & Maher, 2013Tashman, 2010), professional psychology (e.g., Fouad et al, 2009;Gonsalvez et al, 2013;Kaslow et al, 2009;Muse & McManus, 2013;Newell, Newell, & Looser, 2013;Petti, 2008;Schulte & Daly, 2009;Yap, Bearman, Thomas, & Hay, 2012), and medicine (e.g., Andrews, Violato, Ansari, Donnon, & Pugliese, 2013;Dijkstra, van der Vleuten, & Schuwirth, 2009;Epstein, 2007;McMullan et al, 2003;Schuwirth & van der Vleuten, 2011).…”

Validation of Multisource Feedback in Assessing Medical Performance: A Systematic Review

“…The wide adoption of MSF across a multitude of medical disciplines internationally predicates an inherent need to critically evaluate evidence to support or refute its validity. Whilst a number of systematic reviews demonstrate MSF to be a valid, reliable and feasible method of performance assessment [8][9][10][11] , critics regularly cite concerns around important issues that may undermine assessment validity [12][13][14] . Past reviews have largely focussed on specific areas of assessment validity (e.g.…”