Introduction: With the transition of the United States Medical Licensing Examination (USMLE) Step 1 exam to pass-fail, residency directors are exploring alternative objective approaches when selecting candidates for interviews. The Medical Student Performance Evaluations (MSPE) portion of the application may be an area where objectivity could be provided. This study explored program directors’ (PDs) perspectives on the utility of the MSPE as a discriminating factor for residency candidate selection.   Methods: We invited PDs of primary care residencies listed in the American Medical Association FRIEDA database to participate in a mixed-methods study assessing opinions on the MSPE, and the importance of student skills and application components when considering a candidate for interview. We obtained summary statistics for Likert-scale responses. We used inductive thematic analysis to generate themes from open-ended comments.  Results: Two hundred forty-nine PDs completed the survey (response rate=15.9%). Patient communication (83.6%) and teamwork (81.9%) were rated as very/extremely important skills, and being a graduate of a US medical school in the past 3 years (73.1%), no failures on board exams (58.2%), and MSPEs (54.8%) were rated as very/extremely important application components. Six hundred seventy-eight open-ended comments yielded themes related to desire for more transparency and standardization, importance of student attributes and activities, and other important components of applications. Conclusion: PDs place a high value on the MSPE but find it limited by concerns over validity, objectivity, and lack of standardization. The quality of MSPEs may be improved by using a common language of skill attainment such as the Association of American Medical Colleges’ Entrustable Professional Activities and using the document to discuss students’ other attributes and contributions.

Context: As a result of the COVID-19 pandemic, interviews during the 2021 U.S. residency match were conducted virtually; a practice again recommended and repeated by many programs in 2022. The impact of transitioning to virtual interviews on match outcomes is not well studied. If not detrimental, continuing virtual interviewing in the future may be preferred by applicants and programs. Objective: To evaluate the impact of virtual interviews by comparing a baseline of in-person resident interviewing outcomes from 2016-2019 to virtual interviewing outcomes using data from the 2020-2022 virtual interview seasons in three Family Medicine (FM) residency programs. Study Design and Analysis: Retrospective cross-sectional analysis of National Residency Matching Program® data between 2016 -2022 . Aggregate in-person data (2016 -2019 was compared to aggregate virtual data (2020-2022) for each program using chi-square, Fisher Exact test or 2-tailed t-tests to 95% confidence. Setting or Dataset: Three Pennsylvania FM residency programs affiliated with Penn State Health (PSH): the M.S. Hershey Medical center in Hershey, a three-year University-based program; the Mount Nittany Medical Center in State College, a three-year community-based university affiliated program; and Saint Joseph Hospital in Reading, a three-year community-based university affiliated program. Population Studied: FM residency applicants at three programs affiliated with PSH. Intervention/Instrument: Retrospective chart review. Outcome Measures: Fill rate, MD/DO ratio, average position on rank list, average distance from residency site to current and permanent address, and % URIM matched resident. Results: One program had significantly more unfilled positions during virtual recruitment (p=0.006); two had significant differences in the proportion of MD:DO matched applicants (p=0.001; p<0.001), and one program had a significant difference in distance of matched resident's current address (p=.048). Virtual interviews were not associated with significant differences in average position on rank list, average distance from permanent address zip code, or percentage of under-represented in medicine demographic status for matched applicants. Conclusions: The impact of virtual interviewing on FM match result is likely site specific and generally small. Further research is needed to confirm the generalizability of these results to determine major match differences between virtual and in-person interviews.

Background and Objectives: As a result of the COVID-19 pandemic, interviews during the 2021 US residency match were conducted virtually, a practice again recommended and repeated by many programs in 2022. The impact of virtual interviews on recruitment and match outcomes has recently been of interest, with results showing the virtual format to be mostly well received by applicants due to cost, travel, and scheduling benefits. Few studies have looked at pre/posttransition comparisons of applicant geographic and demographic data. We compared objective match outcomes between in-person and virtual interviews across three residency programs. Methods: We conducted a retrospective cross-sectional analysis of National Residency Matching Program data between 2015-2022 across three family medicine residency programs. Primary outcomes were fill rate, average rank position, distance from program, and percentage of underrepresented in medicine demographic status for matched applicants. We compared aggregate in-person data (2015-2019) to aggregate virtual data (2020-2022) for each program using χ2, Fisher Exact test, or 2-tailed t tests to 95% confidence. Results: Saint Joseph Hospital in Reading, Pennsylvania, a 3-year community-based university affiliated program, had significantly more unfilled positions during virtual recruitment (P=.0058). Mount Nittany Medical Center in State College, Pennsylvania, a 3-year community based university-affiliated program, had a significant difference in distance of matched residents’ current address (P=.048). Virtual interviews were not associated with significant differences in average position on rank list, average distance from permanent address zip code, or percentage of underrepresented in medicine (URiM) demographic status for matched applicants. Conclusions: The impact of virtual interviewing on unfilled positions and geographic data is likely site specific and generally small, as some programs had significant structural changes. Further research is needed to confirm the generalizability of these results and explore future comparisons of demographic and geographic characteristics of matched applicants pre/posttransition to the virtual format.