Script concordance test acceptability and utility for assessing medical students’ clinical reasoning: a user’s survey and an institutional prospective evaluation of students’ scores

Background: SCT, considered a valid alternative to the classical evaluation method, is a distinctive method to evaluate medical students’ clinical reasoning skills. The present study aimed to determine the correlation between SCT and clinical-based classical multiple-choice questions (MCQs) in the clinical evaluation of the course of oral disease among Guilan dental students. Methods: In this analytical cross-sectional study, 73 final-year students of the dentistry school of Guilan University who had completed oral and dental diseases courses 1 to 5 were investigated. In this study, in addition to the classical MCQ test at the end of the semester, 15 SCT scenarios with three follow-up questions (45 questions in total) were also administered to the students. The difficulty and discrimination coefficients and the correlation coefficient between the SCT and MCQ tests were measured. Results: The results showed that the mean age of the students was 24.1±59.36, and 53% (39 individuals) were males. Their GPA of the previous semester was 15.42 ± 1.23. Based on the obtained results, a direct and significant relationship was found between the SCT and the MCQ scores (P=0.035); as the SCT scores increased, the MCQ scores increased accordingly. Conclusions: SCT scores were correlated with the results obtained in the classical MCQ test. It is recommended that more research on SCT be used in formative evaluations.

show abstract

“…The students' scores with standardized test methods improved more over time than with SCT. Students' general opinion about SCT was negative (30).…”

Section: Discussionmentioning

confidence: 98%

Assessment of Clinical Reasoning in The Evaluation of the Oral Diseases Course in Final-Year Dental Students Using Script Concordance Test (SCT)

Bashardoust

Mohammadpour

Asgari

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Validity and reliability of Brier scoring for assessment of probabilistic diagnostic reasoning

Stehouwer,

Rowland-Seymour,

Gruppen

et al. 2024

Diagnosis

View full text Add to dashboard Cite

Objectives Educators need tools for the assessment of clinical reasoning that reflect the ambiguity of real-world practice and measure learners’ ability to determine diagnostic likelihood. In this study, the authors describe the use of the Brier score to assess and provide feedback on the quality of probabilistic diagnostic reasoning. Methods The authors describe a novel format called Diagnostic Forecasting (DxF), in which participants read a brief clinical case and assign a probability to each item on a differential diagnosis, order tests and select a final diagnosis. DxF was piloted in a cohort of senior medical students. DxF evaluated students’ answers with Brier scores, which compare probabilistic forecasts with case outcomes. The validity of Brier scores in DxF was assessed by comparison to subsequent decision-making in the game environment of DxF, as well as external criteria including medical knowledge tests and performance on clinical rotations. Results Brier scores were statistically significantly correlated with diagnostic accuracy (95 % CI −4.4 to −0.44) and with mean scores on the National Board of Medical Examiners (NBME) shelf exams (95 % CI −474.6 to −225.1). Brier scores did not correlate with clerkship grades or performance on a structured clinical skills exam. Reliability as measured by within-student correlation was low. Conclusions Brier scoring showed evidence for validity as a measurement of medical knowledge and predictor of clinical decision-making. Further work must evaluated the ability of Brier scores to predict clinical and workplace-based outcomes, and develop reliable approaches to measuring probabilistic reasoning.

show abstract

Script Concordance Test in Physiology: Preparation, Scoring, and Student Perceptions – A Mixed Method Study

Nalini,

Manivasakan,

Pai

2024

International Journal of Applied &Amp; Basic Medical Research

View full text Add to dashboard Cite

Background: Although the curriculum has changed, assessment tools are not in alignment with the new types of teaching such as early clinical exposure (ECE) and self-directed learning. Both in summative and formative assessment most commonly used tools for assessment of cognitive domain are written formats including MCQ. However, these assessment tools such as MCQ and written essays cannot assess the higher order thinking skills and clinical reasoning skills. Materials and Methods: The present study was conducted in the department of physiology, in collaboration with medical simulation center, as a part of formative assessment for topics on pathophysiology of shock and obstructive and restrictive lung disorders taught during ECE using case scenarios. Two script concordance tests (SCTs) each on the topic of pathophysiological mechanism of different types of shock and obstructive and restrictive lung disorders each were prepared and administered to the students. Student perception to this assessment tools was obtained. Results: The Cronbach’s alpha of the 6-item SCT for respiratory and cardiovascular topic administered to the students (n = 107, 98) was 0.83 and 0.82, respectively. The inter-item correlation was respiratory and cardiovascular topic for 0.71, 0.69 suggesting that they are close and repetitive. Students were of the opinion that SCT are difficult to understand (75.8%), challenging to answer (47.2%), and it tested the clinical content better (71.4%). Conclusions: In our study, students felt though SCT tested the clinical content better when compared to conventionally used MCQ, they still find it challenging to understand and are not in favor of its use in summative and formative assessments.

show abstract

Script concordance test acceptability and utility for assessing medical students’ clinical reasoning: a user’s survey and an institutional prospective evaluation of students’ scores

Cited by 4 publications

References 38 publications

Assessment of Clinical Reasoning in The Evaluation of the Oral Diseases Course in Final-Year Dental Students Using Script Concordance Test (SCT)

Assessment of Clinical Reasoning in The Evaluation of the Oral Diseases Course in Final-Year Dental Students Using Script Concordance Test (SCT)

Validity and reliability of Brier scoring for assessment of probabilistic diagnostic reasoning

Script Concordance Test in Physiology: Preparation, Scoring, and Student Perceptions – A Mixed Method Study

Contact Info

Product

Resources

About