Paper 1 conforms to the majority of its stated intentions. However, further development of techniques for investigating validity will be required in order to minimize content variation between both sittings of the examination as well as to help more closely define areas of competence to be tested by Paper 1 of the MRCGP examination.

No abstract

Reliability in written examinations is taken very seriously by examination boards and candidates alike. Within general education many factors influence reliability including variations between markers, within markers, within candidates and within teachers. Mechanisms designed to overcome, or at least minimize, the impact of such variables are detailed. Methods of establishing reliability are also explored in the context of a range of assessment situations. In written tests of general practice within the Membership of the Royal College of General Practitioner (MRCGP) examination considerable effort has put been put into achieving acceptable levels of reliability. Current mechanisms designed to ensure high reliability are described and related to the evolution of the written component of the examination. In addition to description of marker selection and training, question development including construct a detailed example of specific and generic marking schedules is provided. Examination results for the Written Paper of the MRCGP from 1998 to 2003 are reported including Cronbach's alpha coefficients and standard error of measurements, mean scores (and SD) and pass rates. In addition individual discrimination scores for each question in the October 2002 paper are shown. Consistent high reliability of the written component of the MRCGP examination provides valuable lessons in terms of selection, training and monitoring of markers as well as practical methods of moderating factors affecting candidate variability. The challenge for examination developers is to carry these important lessons forward into a modernized assessment structure of UK general practice.

No abstract

No abstract