Structured viva validity, reliability, and acceptability as an assessment tool in health professions education: a systematic review and meta-analysis

Abuzied, Abdelhamid Ibrahim Hassan; Nabag, Wisal Omer Mohamed

doi:10.1186/s12909-023-04524-6

Cited by 4 publications

(1 citation statement)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One way to test the reliability and validity of a student's knowledge is through a viva voce (Abuzied & Nabag, 2023). Just having this as an option may prevent a student from including information that does not come directly from their own thinking.…”

Section: Academic Staff Recommendationsmentioning

confidence: 99%

An attempt to cheat using GPT-4: findings, discussion and recommendations for academic staff and students

Mapletoft,

Price,

Smith

et al. 2024

ETLHE

View full text Add to dashboard Cite

This manuscript explores the use and detection of ChatGPT artificial intelligence (AI) content, in the academic setting. The study involves faculty and student partnership at a Higher Education Provider (HEP), examining the implications of the chatbot's use in academic writing and its potential impact on educational practice. We employ a dialogical approach in this Scholarship of Teaching and Learning (SoTL) study, mirroring the ChatGPT conversational style. A former student uses ChatGPT-4 to rewrite an essay, and the generated content is evaluated by free-to-use AI checking tools, Turnitin, and the module tutor. Our findings illustrate that while ChatGPT-4 could generate a passable assignment, it lacks depth, criticality, and contextualisation. Further, AI checking tools, while useful, exhibit limitations. However, the generated content could serve as a starting point for students, suggesting a possible shift in the educational paradigm with AI's integration. The manuscript also underscores the importance of understanding and navigating AI responsibly, calling for clear guidelines for AI use and improved methods of detecting AI generated content. While AI has emerged as an efficient tool for generating academic content, our study highlights the necessity of human-centric learning and critical thinking skills in academia, emphasising that AI should not replace personal research. We also discuss potential changes to assessment methods in response to AI advancements, recommending strategies for both academic staff and students. Our findings have implications for educational institutions worldwide as AI tools become increasingly accessible, raising questions about academic integrity, pedagogical strategies, and the evolving relationship between technology and education.

show abstract

Section: Academic Staff Recommendationsmentioning

confidence: 99%

An attempt to cheat using GPT-4: findings, discussion and recommendations for academic staff and students

Mapletoft,

Price,

Smith

et al. 2024

ETLHE

View full text Add to dashboard Cite

show abstract

“A pilot trial of objective structured practical examinations (OSPEs) in non-vocational science-based degrees

Karunaratne,

Exintaris,

Zhou

et al. 2024

Innovations in Education and Teaching International

View full text Add to dashboard Cite

Prevalence of Dunning-Kruger effect in first semester medical students: a correlational study of self-assessment and actual academic performance

Knof,

Berndt,

Shiozawa

2024

BMC Med Educ

View full text Add to dashboard Cite

Introduction The ability to self-assess is a crucial skill in identifying one’s own strengths and weaknesses and in coordinating self-directed learning. The Dunning-Kruger effect occurs when limited knowledge causes individuals to overestimate their competence and underestimate others’, leading to poor self-assessment and unrecognized incompetence. To serve as a foundation for developing strategies to improve self-assessment, the self-assessment abilities of first-semester students were assessed. Methods In the final weeks of the summer 2021, winter 2021/22, and summer 2022 semesters, the academic performance (oral anatomy exam) of first semester students was assessed (0–15 points). Before the exam results were announced, students were asked to self-assess their performance. Results Exam scores (M = 10.64, SD = 2.95) and self-assessed scores (M = 10.38, SD = 2.54) were comparable. The absolute difference between them, as a measure of self-assessment ability ranged from − 9 to + 9 points (M = -0.26, SD = 2.59). Among participants (N = 426), 18.5% assessed themselves accurately, 35.5% overestimated, and 46.0% underestimated their performance. The correlation between actual score and self-assessment was ρ = -0.590 (p < 0.001), reflecting the Dunning-Kruger effect. When separated by gender, correlation for females was ρ = -0.591 (p < 0.001), and for males ρ = -0.580 (p < 0.001). Conclusions Realistic self-assessment is a challenge for first-semester students. The data indicate that females tend to overestimate their performance while males underestimate theirs. A pronounced Dunning-Kruger effect is evident in both genders, with significant negative correlations between self-assessment and actual performance. There are several reasons for the occurrence of the Dunning-Kruger effect. Considering that the COVID-19 pandemic influenced learning environments, collaborative learning was significantly restricted. The lack of opportunities for comparison could potentially lead to unrealistic self-assessment.

show abstract

Structured viva validity, reliability, and acceptability as an assessment tool in health professions education: a systematic review and meta-analysis

Cited by 4 publications

References 25 publications

An attempt to cheat using GPT-4: findings, discussion and recommendations for academic staff and students

An attempt to cheat using GPT-4: findings, discussion and recommendations for academic staff and students

“A pilot trial of objective structured practical examinations (OSPEs) in non-vocational science-based degrees

Prevalence of Dunning-Kruger effect in first semester medical students: a correlational study of self-assessment and actual academic performance

Contact Info

Product

Resources

About