A Multicenter Analysis of the Ophthalmic Knowledge Assessment Program and American Board of Ophthalmology Written Qualifying Examination Performance

Lee, Andrew G.; Oetting, Thomas A.; Blomquist, Preston H.; Bradford, G. E.; Culican, Susan M.; Kloek, Carolyn E.; Krishnan, Chandrasekharan; Lauer, Andreas K.; Levi, Leah; Naseri, Ayman; Rubin, Steven E.; Scott, Ingrid U.; Tao, Jeremiah; Tuli, Sonal S.; Wright, Martha M.; WuDunn, Darrell; Zimmerman, M. Bridget

doi:10.1016/j.ophtha.2012.06.010

Cited by 24 publications

(21 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The study of Lee et al, which is the largest to date, found that the OKAP year 3 examination was the best indicator of ABO-WQE pass rate; however, according to their model, even those residents who score in approximately the 4th percentile still had a 50% pass rate, i.e., the positive predictive value for failing the ABO-WQE was 50%. 4 Likewise, in our model, we found the highest indicator of failing the ABO-WQE on first attempt was not achieving a score of 20th percentile on the second-year OKAP examination; however, the positive predictive value for failing the ABO-WQE was 47%, meaning 53% residents still went on to pass the ABO-WQE. These numbers seem significant, except that the positive predictive value for failing the ABO-WQE among graduating ophthalmology residents nationwide, independent of performance on OKAP examinations, is 19.7% on first attempt and approximately 30% overall.…”

Section: Discussionmentioning

confidence: 51%

“…A multicenter analysis demonstrated a statistically significant difference in mean USMLE Step 1 scores among residents who passed the ABO-WQE on first attempt and those who failed. 4 When we performed this analysis in our cohort, there was also a statistical difference (p ¼ 0.004) with the mean USMLE Step 1 percentile being 28.6th among residents who failed the ABO-WQE and 75th among those who passed (►Table 3). Johnson et al reported in their study that performance on the USMLE Step 1 correlated to performance on the OKAP examinations in decreasing strength, but there was no correlation to performance on the ABO-WQE.…”

Section: Discussionmentioning

confidence: 83%

“…There have been a few recent studies published investigating correlations between standardized evaluations such as the United States Medical Licensing Exam (USMLE) Step 1, Ophthalmology Knowledge Assessment Program (OKAP) and American Board of Ophthalmology Written Qualifying Examination (ABO-WQE). [2][3][4][5] Some of these studies have suggested a correlation between performance on the OKAP examinations and passing the ABO-WQE, which seemed contrary to the experience at our institution. Therefore, we have set out to perform an analysis in similar fashion to these prior studies among the graduating residents at our institution.…”

mentioning

confidence: 76%

See 2 more Smart Citations

Standardized Training Examinations among Ophthalmology Residents and the American Board of Ophthalmology Written Qualifying Examination First Attempt: The Morsani College of Medicine Experience

Carey

Drucker

2014

Journal of Academic Ophthalmology

View full text Add to dashboard Cite

Medical training programs have a variety of methods for evaluating applicants and residents, including subjective, objective, and standardized systems. However, it can often be difficult to foresee which applicants will be successful in a residency program and which residents will be successful physicians. To help academic institutions measure resident AbstractPurpose The aim of the study is to demonstrate whether resident performance on the United States Medical Licensing Exam (USMLE) Step 1 and the Ophthalmology Knowledge Assessment Program (OKAP) correlates to American Board of Ophthalmology Written Qualifying Examination (ABO-WQE). Design This is a retrospective cohort study. Participants Forty-one residents who completed their postgraduate training between 1999 and 2011 from our institution were included in the study. Methods Percentiles on the USMLE Step 1, OKAP, and ABO-WQE first attempt were recorded and analyzed for possible correlations among scores across examinations using Pearson rank correlation, chi-square analysis, Fisher exact test, and Student t-test. Results The percentile score on the USMLE Step 1 was the only standardized test that had a statistically significant association with the percentile score on the ABO-WQE (Pearson rank coefficient ¼ 0.45, p ¼ 0.01). Not achieving a score of 20th percentile on the OKAP examination among second-year residents had the highest statistical association with failing the ABO-WQE on first attempt (p ¼ 0.001) with an odds ratio of 20.4, and the relative risk of failing the ABO-WQE was 11.3; however, the relative risk of passing if the cutoff was achieved was only 1.8, and a positive predictive value for failing the ABO-WQE on first attempt of 0.47. Conclusions Percentile scores on the OKAP are not indicative of percentile scores on the ABO-WQE. While OKAP examinations in the second and third year appear to separate out residents at risk of failing the ABO-WQE, there is a large disparity between the risk of passing and failing which is likely due to the inadequate prognostic value of the test. It is likely OKAP examinations may be a useful tool to identify residents who are at low risk for failing the ABO-WQE on first attempt, but much less useful for identifying residents who are at high risk for failing the ABO-WQE.

show abstract

Section: Discussionmentioning

confidence: 51%

Section: Discussionmentioning

confidence: 83%

mentioning

confidence: 76%

See 1 more Smart Citation

Standardized Training Examinations among Ophthalmology Residents and the American Board of Ophthalmology Written Qualifying Examination First Attempt: The Morsani College of Medicine Experience

Carey

Drucker

2014

Journal of Academic Ophthalmology

View full text Add to dashboard Cite

show abstract

“…Some studies, like ours, found middle-oftraining scores to have the greater predictive power. 3,14,17 Others found that predictive power of SAE/ITE scores increased with each year of training.…”

Section: Discussionmentioning

confidence: 99%

“…This was found to be the case in family medicine, 1,2 internal medicine, 3 psychiatry, 4,5 radiology, 6 anesthesiology, 7,8 neurology, 9 obstetrics and gynecology, 10 pediatrics, 11 surgery, 12 orthopedic surgery, [13][14][15] ophthalmology, 16,17 urology, 18 pathology, 19 preventive medicine, 20 otolaryngology, 21 and emergency medicine (osteopathic). 22 Data are available from 3 published and 2 unpublished studies attempting to predict the American Board of Physical Medicine and Rehabilitation (ABPMR) examination performance.…”

Section: Introductionmentioning

confidence: 99%

Predicting Performance on the American Board of Physical Medicine and Rehabilitation Written Examination Using Resident Self-Assessment Examination Scores

Moroz

Bang

2016

Journal of Graduate Medical Education

View full text Add to dashboard Cite

show abstract

Implicit Bias and the Association of Redaction of Identifiers With Residency Application Screening Scores

et al. 2021

View full text Add to dashboard Cite

IMPORTANCE Diversity in the ophthalmology profession is important when providing care for an increasingly diverse patient population. However, implicit bias may inadvertently disadvantage underrepresented applicants during resident recruitment and selection.OBJECTIVE To evaluate the association of the redaction of applicant identifiers with the review scores on ophthalmology residency applications as an intervention to address implicit bias. DESIGN, SETTING, AND PARTICIPANTSIn this quality improvement study, 46 faculty members reviewed randomized sets of 462 redacted and unredacted applications from a single academic institution during the 2019-2020 ophthalmology residency application cycle.INTERVENTIONS Applications electronically redacted for applicant identifiers, including name, sex or gender, race and ethnicity, and related terms. MAIN OUTCOMES AND MEASURESThe main outcome was the distribution of scores on redacted and unredacted applications, stratified by applicant's sex, underrepresentation in medicine (URiM; traditionally comprising American Indian or Alaskan Native, Black, and Hispanic individuals) status, and international medical graduate (IMG) status; the application score β coefficients for redaction and the applicant and reviewer characteristics were calculated. Applications were scored on a scale of 1 to 9, where 1 was the best score and 9 was the worst score. Scores were evaluated for a significant difference based on redaction among female, URiM, and IMG applicants. Linear regression was used to evaluate the adjusted association of redaction, self-reported applicant characteristics, and reviewer characteristics with scores on ophthalmology residency applications. RESULTSIn this study, 277 applicants (60.0%) were male and 71 (15.4%) had URiM status; 32 faculty reviewers (69.6%) were male and 2 (0.4%) had URiM status. The distribution of scores was similar for redacted vs unredacted applications, with no difference based on sex, URiM status, or IMG status. Applicant's sex, URiM status, and IMG status had no association with scores in multivariable analysis (sex, β = -0.08; 95% CI, -0.32 to 0.15; P = .26; URiM status, β = -0.03; (95% CI, -0.36 to 0.30; P = .94; and IMG status, β = 0.39; 95% CI, -0.24 to 1.02; P = .35). In adjusted regression, redaction was not associated with differences in scores (β = −0.06 points on a 1-9 scale; 95% CI, -0.22 to 0.10 points; P = .48). Factors most associated with better scores were attending a top 20 medical school (β = −1.06; 95% CI, -1.37 to -0.76; P < .001), holding an additional advanced degree (β = −0.86; 95% CI, -1.22 to -0.50; P < .001), and having a higher United States Medical Licensing Examination Step 1 score (β = −0.35 per 10-point increase; 95% CI, -0.45 to -0.26; P < .001). CONCLUSIONS AND RELEVANCEThis quality improvement study did not detect an association between the redaction of applicant characteristics on ophthalmology residency applications and the application review scores among underrepresented candidates at this institution. Although the s...

show abstract

A Multicenter Analysis of the Ophthalmic Knowledge Assessment Program and American Board of Ophthalmology Written Qualifying Examination Performance

Cited by 24 publications

References 6 publications

Standardized Training Examinations among Ophthalmology Residents and the American Board of Ophthalmology Written Qualifying Examination First Attempt: The Morsani College of Medicine Experience

Standardized Training Examinations among Ophthalmology Residents and the American Board of Ophthalmology Written Qualifying Examination First Attempt: The Morsani College of Medicine Experience

Predicting Performance on the American Board of Physical Medicine and Rehabilitation Written Examination Using Resident Self-Assessment Examination Scores

Implicit Bias and the Association of Redaction of Identifiers With Residency Application Screening Scores

Contact Info

Product

Resources

About