No abstract

Registrars in general practice have to submit an audit project as one of four parts of summative assessment. A criterion-referenced marking schedule has been developed in the West of Scotland, consisting of five independent criteria all of which have to be judged above minimum competence to pass. A system was developed to test the instrument using a marking exercise which calculated the sensitivity and specificity of the assessment process, for different combinations of assessors. One hundred and two registrar audit projects were then assessed by three independent assessors. Ninety-two (90%) passed and 10 projects (10%) were referred back to the registrar as being below minimum competence. After resubmission six projects (6%) passed, two projects (2%) were still below minimum competence, and two (2%) were not resubmitted. A referral process for assessing the audit projects of general practice registrars has been developed to maximize the opportunity of finding a project below minimum competence.

Evidence from the pilot project has shown that a general practice registrar's ability to review and critically analyse a piece of his/her work, with appropriate management of any necessary change, can be tested feasibly by means of a completed audit cycle within the registrar year. The process retains adequate levels of sensitivity and specificity and requires fewer assessors for marking the projects.

An audit project has been submitted by all trainees in the West of Scotland since 1992 as part of a pilot process for summative assessment. The impact of 2 consecutive years of audit was assessed on 117 trainees in May 1994. A response rate of 89% was achieved. For 82 trainees (79%) this was their first practical experience of audit and as a result of it 85% felt more confident in introducing change to their next practice. Protected time was still a problem for the majority (53%) and one-third wanted more help from their trainer. Thirty trainees (29%) had attended four or fewer formal practice meetings in their 10 months of training, with 10% never having attended one. The previous trainee's audit project was rarely or never discussed with 70 trainees (87%) and only 12 trainees were evaluating changes recommended. An audit project as part of summative assessment may be encouraging trainees to think about change after they leave their training practices. Few, however, are evaluating change and protected time and support are still required. The marking of the summative assessment audit project may need to address this.

Little is known about the audit activity taking place among general practitioners as part of their vocational training. All 155 trainers and their trainees in the West of Scotland were asked about their attitudes to and their experiences of audit. Two hundred and thirty-five replies were received from 310 questionnaires sent, giving an 85% response from trainers and 67% from trainees. Both trainers and trainees had positive attitudes to audit with regard to its use: for assessing work (87% of trainers, 97% trainees); as an appropriate use of resources (92% of trainers, 78% of trainees); as an appropriate use of time (91% trainers, 76% trainees); in improving patient care (96% of trainers, 76% of trainees). Most trainers (90%) and 52% of trainees had started collecting data with a view to starting an audit, 56% of trainers and 23% of trainees had personally set a standard and 54% of trainers but only 12% of trainees had completed a cycle of audit. Despite very positive attitudes to audit there is a problem completing a cycle of audit, particularly among trainees. They will thus be denied the benefit of negotiating and evaluating change as part of their training, skills which will be necessary to ensure a lifelong awareness of the quality of care they are providing.