Analyzing Script Concordance Test Scoring Methods and Items by Difficulty and Type

Abstract

Of the 6 scoring methods, 5-point scoring solutions generated more reliable measures of data interpretation than 3-point scoring methods. Data interpretation abilities were a function of experience at every level of item difficulty. Items categorized by type exhibited discriminatory power providing modest evidence toward the construct validity of SCTs.

The cognitive process of test takers when using the script concordance test rating scale

“…Several other formats have been proposed in the existing literature. For example, the use of a 3‐point Likert scale has been investigated, but this may reduce the discriminatory power of the SCT . Alternatively, the use of different rating scale descriptors (ie, much more/less probable/useful/appropriate) has been considered, but this may result in different interpretations by test takers.…”

“…For example, the use of a 3-point Likert scale has been investigated, 41 but this may reduce the discriminatory power of the SCT. 42 Alternatively, the use of different rating scale descriptors (ie, much more/less probable/useful/appropriate) has been considered, 43 The innumerable possible modifications to the numerical scale and the resulting impact on the cognitive process of response matching suggests that further study of response process validity evidence is required to support the use of the SCT in CDM skills assessment.…”

Assessing clinical reasoning in optometry using the script concordance test

“…Moreover, the distribution of answers was similar between the students and the members of the reference panel (Table ), so this is unlikely to have had much effect on the results of this study. Research investigating optimal methods for selecting scoring and standard setting for script concordance tests is on‐going …”

“…Research investigating optimal methods for selecting scoring and standard setting for script concordance tests is on-going. [42][43][44] Since this first experience with a script concordance test in optometry has been shown to be conclusive, the next step will be expanding the bank of clinical scenarios. This means, among others, contemplating the creation of more scenarios with visual content (visual field, topography, imaging, photographs, videos).…”

Implementation of an education‐focused PhD program in anatomy and cell biology at Indiana University: Lessons learned and future challenges

“…As of June 2014, the Education Track students had authored or coauthored ten journal articles (Barger, ; Keller and Schutte, ; Burr and Guillot, ; Collier et al, ; Cooper et al, ; Foo et al, ; Schutte and Braun, ; Fillmore et al, ; Wilson et al, , b). These students also have made over 30 presentations at meetings of the AAA, American Association of Clinical Anatomists (AACA), Human Anatomy and Physiology Society (HAPS), and other professional organizations.…”