Multisource feedback as part of the Medical Board of Australia’s Professional Performance Framework: outcomes from a preliminary study

Abstract

BackgroundThe recent introduction of the Professional Performance Framework by the Medical Board of Australia is intended to strengthen continuing professional development for the 100,000 or so medical practitioners in Australia. An important option within the Framework is the use of multisource feedback from patients, colleagues and self-evaluations to allow doctors to reflect on their performance and identify methods for self-improvement. The aim of this study is to explore the relationships between patient feedback, colleague feedback, and self-evaluation using the same questionnaires as used by patients and colleagues.MethodsFeedback data for around 2000 doctors belonging to four different groups were collected through non-probability sampling from nearly 100,000 patients and 24,000 colleagues. Reliability analysis was performed using single measures intraclass coefficients, Cronbach’ alpha and signal-to-noise ratios. Analysis of variance was used to identify significant differences in scores between items and sub-populations of doctors; principal component analysis involving Kaiser-Meyer-Olkin (KMO) sampling adequacy and Bartlett’s test for sphericity was used to reveal components of doctor performance; and correlation analysis was used for identifying convergence between sets of scores from different sources.ResultsPatients rated doctors highest on respect shown and lowest on reassurance provided. Colleagues rated doctors highest on trustworthiness and lowest on ability to say ‘no’. With regard to self-evaluation, doctors gave themselves lower scores on the patient questionnaire and the colleague questionnaire (10 and 12%, respectively) than they received from their patients and colleagues. There were weak but positive correlations between self-scores and scores received indicating some convergence of agreement, with doctors feeling more comfortable with self-evaluation from the perspective of patients than from colleagues.ConclusionsSupplementing patient and colleague feedback with self-evaluation may help doctors confirm for themselves areas for enhanced CPD through convergence. If self-evaluation is used, the colleague questionnaire may be sufficient, since aspects of clinical competence, management, communication and leadership as well as patient care can be addressed through colleague items. Mentoring of doctors in CPD should aim to make doctors feel more comfortable about being rated by colleagues to enhance convergence between self-scores and evaluations from the perspective of colleagues.

Colleague appraisal of Australian general practitioners in training: an analysis of multisource feedback data

“…Participating GPiT were advised to nominate at least 15 colleagues with whom they work, including doctors, other healthcare professionals and managerial/administrative staff [ 1 ]. Nominated colleagues were then sent the questionnaire (Colleague Feedback Evaluation Tool [ 1 , 2 ]) for completion, with a follow-up reminder, if required. The colleague questionnaire asks colleagues to rate their interactions with the target doctor on aspects of clinical competence, management, communication and leadership [ 1 ] There is a final question relating to overall ability.…”

“…Multisource feedback (MSF) is a valued educational feedback and formative assessment tool used to facilitate reflection on communication skills, teamwork, and professionalism [ 1 – 3 ]. MSF comprises patient and colleague feedback and self-appraisal, using reliable and validated measures [ 1 , 2 ]. The Royal Australian College of General Practitioners (RACGP) pathways to General Practitioner (GP) Fellowship (i.e., satisfactorily completing the education/training pathway to become a vocationally registered specialist in General Practice) include two different programs.…”

“…MSF comparisons are important to enable an understanding of an individual’s performance with respect to their peers, as well as how different cohorts of doctors compare. For example, Narayanan et al [ 2 ] have demonstrated that doctors who undertook MSF as remediation required by the Australian Health Practitioner Regulation Agency (AHPRA) received lower ratings for their professionalism from colleagues than other groups of GPs, including AGPT GPiT. There is also evidence that colleague feedback scores can be predictive of the future need for remediation and summative assessment performance for GPiT [ 9 , 10 ].…”

The Effect of Post-Graduate Year Training on the Self-Efficacy and Emotional Traits of Physicians Facing the COVID-19 Pandemic

“…The exploratory factor analysis was done to reduce the factor dimension via the varimax rotation method [ 27 , 28 ]. Kaiser–Meyer–Olkin (KMO), a test to determine sample adequacy, and Bartlett’s test of sphericity, a test to determine the degree of interrelations between variables, were used to confirm whether the data were suitable for factor analysis [ 29 , 30 ] under the following conditions: KMO value > 0.7, Bartlett’s test of sphericity p -value < 0.001, and Eigenvalue > 1. Cronbach’s alpha coefficient was used as an estimate of the internal consistency of the questionnaire.…”

The Impact of implementing multisource feedback on behaviors of young doctors

“…Multisource feedback has four basic components "senior, junior, peer colleague, and self-assessment". 1 MSF is being used as a tool for assessment of professional development of health professionals in all over the world [2][3][4] Giving appropriate feedback is an important component of MSF which allows the health care professionals to reflect their performance and improvement. 5 The basic catalyst for change is negative or discrepant feedback.…”