The importance of USMLE step 2 on the screening and selection of applicants for general surgery residency positions

Khalil, Sarah I.; Jose, Joslyn; Welter, Matthew; Timmons, Jennifer; Miller, Lisa; Elian, Alain; Munene, Gitonga; Sawyer, Robert G.; Shebrain, Saad

doi:10.1016/j.heliyon.2023.e17486

Cited by 4 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Licensing Examination and the National Medical Practitioners Qualifying Examination in Japan [11,12]. These examinations serve as essential gateways for assessing the competence of prospective healthcare practitioners, particularly their ability to provide ethical patient care [13].…”

Section: Licentiate Examinations Are Globally Present With Prominent ...mentioning

confidence: 99%

Awareness and Perception of Dental Undergraduates in Belagavi District Towards the Newly Proposed National Exit Test for India

Santhosh,

Coutinho,

Ankola

et al. 2024

Cureus

View full text Add to dashboard Cite

Background: The National Dental Commission Bill of 2023 introduced the National Exit Test (NExT) as a common final examination for all dental graduates in India, granting them the license to practice. This study evaluated dental undergraduates' awareness and perceptions of the newly proposed NExT. Methods: A self-administered questionnaire in English comprising 23 close-ended questions was used to assess the awareness and perception of the students. A pilot study was conducted to determine the sample size, and 510 students were selected using a simple random sampling technique. The survey was administered to students ranging from 1st to 4th-year undergraduates and interns from two dental colleges in the Belagavi District, India. The questionnaire demonstrated good reliability (Cronbach's alpha coefficient = 0.86) and a content validity ratio 0.82. Results: Interns had the highest mean awareness (39.56 ± 8.99) and perception (40.87 ± 5.56) scores, whereas first-year students had the lowest, with statistically significant differences among the groups (p ≤ 0.001). Although 81% of students were aware of NExT in India, only 17.3% found it student-friendly. A positive correlation was seen between the perception and awareness scores (r = + 0.242; p ≤ 0.001). The dependence of awareness and perception scores on predictors such as age, gender, and year of study were 16.7% and 15.3%, respectively. Conclusion: Interns displayed a positive perception and higher awareness of NExT, whereas first and second-year dental students exhibited lower awareness and apprehensive perceptions. The introduction of NExT promises to enhance the overall quality of dental education on a national scale by providing high-quality care to patients.

show abstract

Section: Licentiate Examinations Are Globally Present With Prominent ...mentioning

confidence: 99%

Awareness and Perception of Dental Undergraduates in Belagavi District Towards the Newly Proposed National Exit Test for India

Santhosh,

Coutinho,

Ankola

et al. 2024

Cureus

View full text Add to dashboard Cite

show abstract

“…These scores serve as the primary objective and quantifiable metric, thereby acting as a key benchmark for program directors when evaluating and comparing applicants. Since the transition of USMLE Step 1 to a pass/fail system in 2022, the scores achieved in the USMLE Step 2CK are now considered even more important in the application process 12 , 13 . Interestingly, Kracaw et al showed that across different academic performance measures, the USMLE Step 1 score showed the highest correlation with USMLE Step 2CK scores.…”

Section: Introductionmentioning

confidence: 99%

In-depth analysis of ChatGPT’s performance based on specific signaling words and phrases in the question stem of 2377 USMLE step 1 style questions

Knoedler,

Hoch

et al. 2024

Sci Rep

View full text Add to dashboard Cite

ChatGPT has garnered attention as a multifaceted AI chatbot with potential applications in medicine. Despite intriguing preliminary findings in areas such as clinical management and patient education, there remains a substantial knowledge gap in comprehensively understanding the chances and limitations of ChatGPT’s capabilities, especially in medical test-taking and education. A total of n = 2,729 USMLE Step 1 practice questions were extracted from the Amboss question bank. After excluding 352 image-based questions, a total of 2,377 text-based questions were further categorized and entered manually into ChatGPT, and its responses were recorded. ChatGPT’s overall performance was analyzed based on question difficulty, category, and content with regards to specific signal words and phrases. ChatGPT achieved an overall accuracy rate of 55.8% in a total number of n = 2,377 USMLE Step 1 preparation questions obtained from the Amboss online question bank. It demonstrated a significant inverse correlation between question difficulty and performance with rs = -0.306; p < 0.001, maintaining comparable accuracy to the human user peer group across different levels of question difficulty. Notably, ChatGPT outperformed in serology-related questions (61.1% vs. 53.8%; p = 0.005) but struggled with ECG-related content (42.9% vs. 55.6%; p = 0.021). ChatGPT achieved statistically significant worse performances in pathophysiology-related question stems. (Signal phrase = “what is the most likely/probable cause”). ChatGPT performed consistent across various question categories and difficulty levels. These findings emphasize the need for further investigations to explore the potential and limitations of ChatGPT in medical examination and education.

show abstract

Analyzing Question Characteristics Influencing ChatGPT’s Performance in 3000 USMLE®-Style Questions

Alfertshofer,

Knoedler,

Hoch

et al. 2024

Med.Sci.Educ.

View full text Add to dashboard Cite

Background The potential of artificial intelligence (AI) and large language models like ChatGPT in medical applications is promising, yet its performance requires comprehensive evaluation. This study assessed ChatGPT’s capabilities in answering USMLE® Step 2CK questions, analyzing its performance across medical specialties, question types, and difficulty levels in a large-scale question test set to assist question writers in developing AI-resistant exam questions and provide medical students with a realistic understanding of how AI can enhance their active learning. Materials and Methods A total of n=3302 USMLE® Step 2CK practice questions were extracted from the AMBOSS© study platform, excluding 302 image-based questions, leaving 3000 text-based questions for analysis. Questions were manually entered into ChatGPT and its accuracy and performance across various categories and difficulties were evaluated. Results ChatGPT answered 57.7% of all questions correctly. Highest performance scores were found in the category “Male Reproductive System” (71.7%) while the lowest were found in the category “Immune System” (46.3%). Lower performance was noted in table-based questions, and a negative correlation was found between question difficulty and performance (rs=−0.285, p <0.001). Longer questions tended to be answered incorrectly more often (rs=−0.076, p <0.001), with a significant difference in length of correctly versus incorrectly answered questions. Conclusion ChatGPT demonstrated proficiency close to the passing threshold for USMLE® Step 2CK. Performance varied by category, question type, and difficulty. These findings aid medical educators make their exams more AI-proof and inform the integration of AI tools like ChatGPT into teaching strategies. For students, understanding the model’s limitations and capabilities ensures it is used as an auxiliary resource to foster active learning rather than abusing it as a study replacement. This study highlights the need for further refinement and improvement in AI models for medical education and decision-making.

show abstract

The importance of USMLE step 2 on the screening and selection of applicants for general surgery residency positions

Cited by 4 publications

References 22 publications

Awareness and Perception of Dental Undergraduates in Belagavi District Towards the Newly Proposed National Exit Test for India

Awareness and Perception of Dental Undergraduates in Belagavi District Towards the Newly Proposed National Exit Test for India

In-depth analysis of ChatGPT’s performance based on specific signaling words and phrases in the question stem of 2377 USMLE step 1 style questions

Analyzing Question Characteristics Influencing ChatGPT’s Performance in 3000 USMLE®-Style Questions

Contact Info

Product

Resources

About