Thinking differently – Students’ cognitive processes when answering two different formats of written question

Sam, Amir H.; Wilson, Rebecca; Westacott, Rachel; Gurnell, Mark; Melville, Colin; Brown, Celia

doi:10.1080/0142159x.2021.1935831

Cited by 10 publications

(16 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The attenuation of differences in RM-MCQ first and DA-MCQ first might be explained by cueing and guessing. As VSAQs can largely circumvent cueing and guessing, they provide a better measure of a students’ true content knowledge, which increases validity [2, 8, 14]. The high discriminative capability of VSAQs is further supported by higher average R ir values of VSAQs in DA.…”

Section: Discussionmentioning

confidence: 99%

“…Moreover, feedback suggested that VSAQ phrasing might not always have been clear enough. This led to uncertainty regarding the level of specificity of the desired answer and highlights the importance of a well-designed VSAQ in order to profit from their advantages over MCQs [8, 31].…”

Section: Discussionmentioning

confidence: 99%

“…Nonetheless, as studies on VSAQs have mainly been performed in the UK, there is a need for validation of previous results in different educational systems [2, 3, 8, 14, 18]. Therefore, we aimed to investigate the reliability, discrimination, acceptability, and retention of knowledge of VSAQs compared to MCQs in a cohort of Dutch medical undergraduate students, partially replicating the design of Sam et al .…”

Section: Introductionmentioning

confidence: 98%

“…Assessment in medical education commonly uses multiple choice questions (MCQs). Even though this question type offers high reliability and easy machine-marking, it allows for cueing (i.e., answering questions based on cues in the question or answer options rather than on content knowledge) [1][2][3][4] and stimulates a recognition-based study approach [5][6][7][8]. Although recognition may be sufficient to pass a MCQ-based assessment, it has been critically noted that clinical practice does not offer a multiple choice list of possible diagnoses or procedures, nor is there a single best recognisable answer in the medical profession [9,10].…”

Section: Introductionmentioning

confidence: 99%

“…However, very short answer questions (VSAQs), a free-response type of questions with the answer being limited to 1-4 words, may be better suited to circumvent some of the general limitations of MCQs. The open-ended nature of the VSAQ reflects the clinical practice better and may prevent surface-level study approaches and cueing [1, 3, 4, 8]. In addition, VSAQs are better able to discriminate between students based on proficiency in the content knowledge [1-4, 13, 14] and may increase retention of knowledge [15-17].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Comparison of Very Short Answer Questions and Multiple Choice Questions in Medical Students: Reliability, Discrimination, Acceptability and Effect on Knowledge Retention

Janse

Wijk

Ruijter

et al. 2022

Preprint

View full text Add to dashboard Cite

Introduction: Multiple choice questions (MCQs) offer high reliability and easy machine-marking, but allow for cueing and stimulate recognition-based learning. Very short answer questions (VSAQs) may circumvent these limitations. We investigated VSAQ reliability, discriminative capability, acceptability, and knowledge retention compared to MCQs. Methods: Dutch undergraduate medical students (n=375) were randomised to a formative exam with VSAQs first and MCQs second or vice versa in two courses, to determine reliability and discrimination. Next, acceptability (i.e., VSAQ review time) was determined in the summative exam. Knowledge retention at 2 and 5 months was determined by comparing score increase on the three-monthly progress test (PT) between students tested with VSAQs and students from previous years tested without VSAQs. Results: Reliability (Cronbachs alpha) was 0.74 for VSAQs and 0.57 for MCQs in one course. In the other course, Cronbachs alpha; was 0.87 for VSAQs and 0.83 for MCQs . Discrimination (Rir) was 0.27 vs. 0.17 and 0.43 vs. 0.39 for VSAQs vs. MCQs, respectively. Reviewing time of one VSAQ for the entire student cohort was ± 2 minutes on average. No clear effect on knowledge retention after 2 and 5 months was observed. Discussion: We found increased reliability and discrimination of VSAQs compared to MCQs. Reviewing time of VSAQs was acceptable. The association with knowledge retention was unclear in our study. This study supports and extends positive results of previous studies on VSAQs regarding reliability, discriminative capability, and acceptability in Dutch undergraduate medical students.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Comparison of Very Short Answer Questions and Multiple Choice Questions in Medical Students: Reliability, Discrimination, Acceptability and Effect on Knowledge Retention

Janse

Wijk

Ruijter

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Knowledge and attitude factors influencing primary care clinicians' diagnosis, treatment and management of urinary tract infections in women: A qualitative ‘think’ aloud study in England

Mwape

Schmidtke

Brown

2023

Repro Female Child Health

Self Cite

View full text Add to dashboard Cite

ObjectivesThe present study aimed to identify factors related to knowledge and attitudes that influence primary care clinicians' decision‐making when diagnosing, treating, and managing suspected urinary tract infections (UTIs) in women. Understanding the factors that influence clinicians' decision‐making is important for maximising health outcomes in women and reducing suboptimal use of antibiotics and antimicrobial resistance.DesignA qualitative think‐aloud study design was employed.MethodsSemi‐structured qualitative interviews were conducted with primary care clinicians in England over Microsoft Teams. Interviews were transcribed and coded in two ways. First, clinicians' responses for each scenario were coded as either following (optimal), not following evidence‐based national guideline or incorrectly citing certain diagnosis, treatment, and management procedures (suboptimal). Second, the knowledge and attitude factors that influenced decision‐making were coded according to an empirically informed umbrella framework. Clinicians external to the study team reviewed the findings to promote their trustworthiness and utility.ResultsTen clinicians (six female) took part. Clinicians prescribing decisions were mostly influenced by knowledge rather than attitude factors. Despite clinicians expressing high awareness of relevant evidence‐based guidelines (a knowledge factor) and high confidence (an attitude factor), more than half of their decisions (60%) were suboptimal in some way thereby impacting women's health. Most clinicians relied on their experiential knowledge rather than using evidence‐based guidelines. Our results suggest that knowledge could impede adherence, for example, where local guidelines conflict with national guidelines of when to perform a urine dipstick test.ConclusionsSuboptimal prescribing decisions could result from a combination of different knowledge and attitude factors leading to negative outcomes in women's health such as inappropriate treatment of asymptomatic bacteriuria in women over 65 years old. To optimise antibiotic prescribing in women, policy‐level interventions could increase concordance across local and national guidelines, or more tailored individual‐level interventions could help clinicians recognise where their experiential knowledge causes deviations from evidence‐based guidelines when diagnosing, treating and managing UTIs.

show abstract

The impact of patient skin colour on diagnostic ability and confidence of medical students

Dodd

Rafi

Stackhouse

et al. 2023

Adv in Health Sci Educ

Self Cite

View full text Add to dashboard Cite

Previous literature has explored unconscious racial biases in clinical education and medicine, finding that people with darker skin tones can be underrepresented in learning resources and managed differently in a clinical setting. This study aimed to examine whether patient skin colour can affect the diagnostic ability and confidence of medical students, and their cognitive reasoning processes. We presented students with 12 different clinical presentations on both white skin (WS) and non-white skin (NWS). A think aloud (TA) study was conducted to explore students’ cognitive reasoning processes (n = 8). An online quiz was also conducted where students submitted a diagnosis and confidence level for each clinical presentation (n = 185). In the TA interviews, students used similar levels of information gathering and analytical reasoning for each skin type but appeared to display increased uncertainty and reduced non-analytical reasoning methods for the NWS images compared to the WS images. In the online quiz, students were significantly more likely to accurately diagnose five of the 12 clinical presentations (shingles, cellulitis, Lyme disease, eczema and meningococcal disease) on WS compared to NWS (p < 0.01). With regards to students’ confidence, they were significantly more confident diagnosing eight of the 12 clinical presentations (shingles, cellulitis, Lyme disease, eczema, meningococcal disease, urticaria, chickenpox and Kawasaki disease) on WS when compared to NWS (p < 0.01). These findings highlight the need to improve teaching resources to include a greater diversity of skin colours exhibiting clinical signs, to improve students’ knowledge and confidence, and ultimately, to avoid patients being misdiagnosed due to the colour of their skin.

show abstract

Thinking differently – Students’ cognitive processes when answering two different formats of written question

Cited by 10 publications

References 19 publications

Comparison of Very Short Answer Questions and Multiple Choice Questions in Medical Students: Reliability, Discrimination, Acceptability and Effect on Knowledge Retention

Comparison of Very Short Answer Questions and Multiple Choice Questions in Medical Students: Reliability, Discrimination, Acceptability and Effect on Knowledge Retention

Knowledge and attitude factors influencing primary care clinicians' diagnosis, treatment and management of urinary tract infections in women: A qualitative ‘think’ aloud study in England

The impact of patient skin colour on diagnostic ability and confidence of medical students

Contact Info

Product

Resources

About