We have reviewed over 60 years of studies on healthcare education outreach programs that are aimed to support first-generation, low-income, as well as underrepresented racial and ethnic minority groups (historically marginalized students) to pursue pre-health professions. As a systematic literature review, we present the challenges studies on healthcare education outreach programs had as three main categories: 1) Design, 2) Evaluation, and 3) Analysis. 1) Designs of studies on healthcare education outreach programs often lacked theoretical foundations whereby a) the interventions did not present theories underlying a causal mechanism of inequity in health professions; and/or 1b) the defined outcome measures were not clearly aligned with the problem the intervention tried to address. 2) Evaluations of studies on healthcare education outreach programs were not always conducted effectively whereby: 2a) controlled groups were commonly absent for comparison with the intervention group; and/or 2b) post measures were solely used without pre-measures. 3) Analyses of studies on healthcare education outreach programs were not adequate whereby: 3a) the response rates and effect size were commonly low; and/or 3b) qualitative results commonly did not supplement quantitative results. Overall, our findings reveal studies on healthcare education outreach programs have common challenges that hinder the reliability of their effects supporting historically marginalized students in pursuing pre-health professions. To address such challenges with studies on healthcare education outreach programs aimed at supporting historically marginalized students, we created a decision flow chart for researchers to ask themselves: 1) how is the design guided by theoretical goals; 2) how are measurements used to evaluate success; and 3) how does the analysis lead to reliable results?

Background Trainee supervision and teaching are distinct skills that both require faculty physician competence to ensure patient safety. No standard approach exists to teach physician supervisory competence, resulting in variable trainee oversight and safety threats. The Objective Structured Teaching Evaluation (OSTE) does not adequately incorporate the specific skills required for effective supervision. To address this continuing medical education gap, the authors aimed to develop and identify validity evidence for an “Objective Structured Supervision Evaluation” (OSSE) for attending physicians, conceptually modeled on the historic OSTE. Methods An expert panel used an iterative process to create an OSSE instrument, which was a checklist of key supervision items to be evaluated during a simulated endotracheal intubation scenario. Three trained “standardized residents” scored faculty participants' performance using the instrument. Validity testing modeled a contemporary approach using Kane's framework. Participants underwent simulation‐based mastery learning (SBML) with deliberate practice until meeting a minimum passing standard (MPS). Results The final instrument contained 19 items, including three global rating measures. Testing domains included supervision climate , participant control of patient care , trainee evaluation , instructional skills , case‐specific measures , and overall supervisor rating . Reliability of the assessment tool was excellent (ICC range 0.84–0.89). The assessment tool had good internal consistency (Cronbach's α = 0.813). Out of 24 faculty participants, 17 (70.8%) met the MPS on initial assessment. All met the MPS after SBML and average score increased by 19.5% (95% CI of the difference 10.3%–28.8%, p = 0.002). Conclusions The OSSE is a promising tool to assess faculty supervision performance. Further study should evaluate the effect of OSSE training on supervision in the clinical environment, which may have measurable changes in patient‐centered outcomes and resident teaching.