Paper trials: a qualitative study exploring the place of portfolios in making revalidation recommendations for Responsible Officers

Abstract

BackgroundA portfolio of supporting information (SI) reflecting a doctor’s entire medical practice is now a central aspect of UK appraisal for revalidation. Medical revalidation, introduced in 2012, is an assessment of a doctor’s competence and passing results in a five yearly license to practice medicine. It assesses of a doctor’s professional development, workplace performance and reflection and aims to provide assurance that doctors are up-to-date and fit to practice. The dominant assessment mechanism is a portfolio. The content of the revalidation portfolio has been increasingly prescribed and the assessment of the SI is a fundamental aspect of the appraisal process which ultimately allows Responsible Officers (ROs) to make recommendations on revalidation. ROs, themselves doctors, were the first to undergo UK revalidation. This qualitative study explored the perceptions of ROs and their appraisers about the use of this portfolio of evidence in a summative revalidation appraisal.Methods28 purposefully sampled London ROs were interviewed following their revalidation appraisal and 17 of their appraisers participated in focus groups and interviews. Thematic analysis was used to identify commonalities and differences of experience.ResultsSI was mostly easy to provide but there were challenges in gathering certain aspects. ROs did not understand in what quantities they should supply SI or what it should look like. Appraisers were concerned about making robust judgements based on the evidence supplied. A lack of reflection from the process of collating SI and preparing for appraisal was noted and learning came more from the appraisal interview itself.ConclusionsMore explicit guidance must be available to both appraisee and appraiser about what SI is required, how much, how it should be used and, how it will be assessed. The role of SI in professional learning and revalidation must be clarified and further empirical research is required to examine how best to use this evidence to make judgments as part of this type of appraisal.

Reflective Practice for Patient Benefit: An Analysis of Doctors' Appraisal Portfolios in Scotland

“…There is an indication that patient feedback can be rather anodyne and, in this sample of portfolios, provided little useful feedback to work with; colleague feedback seemed to provide more useful and targeted feedback. This echoes other findings: a survey of ROs found that they considered participation in QI and responses to significant events as the most effective methods of improving doctor performance; 28 the large online survey conducted by UMbRELLA found CPD was the most commonly reported change as a result of appraisal; 17 another survey of GPs also cited QI and SEAs as important in prompting changes, with colleague feedback also regarded as important in helping deeper reflection on their work (though it should be noted that they valued patient feedback almost as much). 29 The latest GMC guidance emphasises the central role of reflection in appraisal and describes it in terms of a twin process, stating that: "your appraiser can facilitate further reflection, as needed but it is your responsibility to demonstrate examples of your reflective practice."…”

Doctors' attitudes to maintenance of professional competence: A scoping review

“…A few studies, however, did find that doctors perceived value from participation in quality improvement activities 80,92,114,122,123,148,149 . Written reflections and patient feedback, although developmental, were two further elements of MPC that doctors did not appreciate 25,91,105,106,108,130,145,146,150,151 . Doctors questioned the reliability of patient feedback tools used in MPC to accurately reflect a doctor's performance and identified barriers to the process of collecting patient feedback 44,51,108,115,130,145,146,150,151 …”

Implementing and evaluating an e-portfolio for postgraduate family medicine training in the Western Cape, South Africa

“…Furthermore, it has been strongly argued that assessment is an educational design problem that needs a programmatic approach [40]. While a portfolio of learning has become standard assessment practice in many programmes worldwide, especially in workplace-based assessment and revalidation, it remains a challenge to provide sufficient evidence of performance or competence [15, 41, 42]. In order to move from assessing performance to assessing professionalism, the next level in Miller’s revised pyramid [43], there is a need to include more methods that rely on qualitative information [40], which imply professional judgement and are most likely incorporated in improved feedback.…”