Association Between Licensing Examination Scores and Resource Use and Quality of Care in Primary Care Practice

Abstract

Context.-Clinical competence is a determinant of the quality of care delivered, and may be associated with use of health care resources by primary care physicians. Clinical competence is assumed to be assessed by licensing examinations, yet there is a paucity of information on whether scores achieved predict subsequent practice. Objective.-To determine if licensing examination scores were associated with selected aspects of quality of care and resource use in initial primary care practice. Design.-Prospective cohort study of recently licensed family physicians, followed up for the first 18 months of practice. Setting.-The Québec health care system. Participants.-A total of 614 family physicians who passed the licensing examination between 1991 and 1993 and entered fee-for-service practice in Québec. Main Outcome Measures.-All patients seen by physicians were identified by the universal health insurance board and all health services provided to these patients were retrieved for the 18 months prior to (baseline) and after (follow-up) the physicians' entry into practice. Medical service and prescription claims files were used to measure rates of resource use (specialty consultation, symptom-relief prescribing compared with disease-specific prescribing) and quality of care (inappropriate prescribing, mammography screening). Baseline data were used to adjust for differences in practice population. Results.-Study physicians saw a total of 1 116 389 patients, of whom 113 535 (10.2%) were elderly and 83 391 (7.5%) were women aged 50 to 69 years. Physicians with higher licensing examination scores referred more of their patients for consultation (3.8/1000 patients per SD increase in score; 95% confidence interval [CI], 1.2-7.0; P = .005), prescribed to elderly patients fewer inappropriate medications (−2.7/1000 patients per SD increase in score; 95% CI, −4.8 to −0.7; P = .009) and more disease-specific medications relative to symptom-relief medications (3.9/1000 patients per SD increase in score; 95% CI, 0.3 to 7.4; P = .03), and referred more women aged 50 to 69 years (6.6/1000 patients per SD increase in score; 95% CI, 1.2-11.9; P = .02) for mammography screening. If patients of physicians with the lowest scores had experienced the same rates of consultation, prescribing, and screening as patients of physicians with the highest scores, an additional 3027 patients would have been referred, 179 fewer elderly patients would have been prescribed symptom-relief medication, 912 more elderly patients would have been prescribed disease-specific medication, 189 fewer patients would have received inappropriate medication, and 121 more women would have received mammography screening. Conclusions.-Licensing examination scores are significant predictors of consultation, prescribing, and mammography screening rates in initial primary care practice.

Impact of performance in a mandatory postgraduate surgical examination on selection into specialty training

“…Several groups have assessed the predictive validity of other high‐stakes medical examinations, including the United States Medical Licensing Examination ® (USMLE ® )8, 9, 10, the Quebec Licensing Examination11, the Medical Council of Canada Qualifying Examination12, 13, 14, the MRCP examination15, 16, 17 and Membership of the Royal College of General Practitioners examination15. Performance in these examinations predicts performance in clinical practice and other medical assessment processes11 18, 19.…”

“…Performance in these examinations predicts performance in clinical practice and other medical assessment processes11 18, 19. For example, each part of the MRCP examination predicts the next17, and candidates who score higher in all parts of the MRCP examination do better in workplace‐based assessments than those who underperform16.…”

Review article: Closing the research gap at the interface of learning and clinical practice

“…8 Eva and Lingard's recommendations are to aggregate forces and skills in research centres, and to view scientific progress not as the single important study but as the multi-year program of research that allows broader exploration of topics. 5 Over multiple years and studies, Tamblyn et al documented the relationship between scores on the Canadian provincial licensing examination and attributes of physicians' performance in practice, [95][96][97][98][99] and in their seminal work, they have shown that a focused body of high-quality research can create a scientific foundation for medical education and assessment. A future cohesive body of research to close the gap at the interface of medical education and practice could be similar to the research in hematology/oncology being organized around the Cancer and Leukemia Group B as a way to overcome the limitations of small local samples of cancer patients through pooled data and collaborative research for more than four decades.…”

Assessing Accomplished Teaching

“…Furthermore, should the outcomes for a high-priced lawyer, who can select his or her clients, be compared to the outcomes for a public defender? While data are available that might be used in such evaluations (e.g., rates of death or guilty verdicts) and several such studies have been conducted (e.g., Norcini et al, 2002;Tamblyn et al, 1998Tamblyn et al, , 2002, many factors can contribute to the outcomes, making interpretation of the relationships very tricky.…”

“…Although there are a few situations in which such evidence has been collected (e.g., Norcini, Lipner, and Kimball, 2002;Tamblyn et al, 1998Tamblyn et al, , 2002, in most cases, no adequate criterion is available, and, in practice, the outcomes depend on many variables beyond the competence of the individual practitioner. Even the best driver can get into accidents, and even the best neurologist will not be successful in every case.…”