Reliability of Online Simulation-Based Assessment to Measure Cognitive Performance and Its Acceptance Among Pharmacy Students

Abstract

The majority of conventional evaluations of cognitive skills (CSs) utilise paper-based or online multiple-choice questions or single best-answer questions. However, examination that uses online simulation of real scenarios has the potential to complement medium-to-high level CSs that make use of the intellectual capabilities of applying, analysing and evaluating complex information. Nevertheless, the reliability of online simulation as an alternative learning and assessment tool for Malaysian pharmacy courses has never been investigated. This paper illustrates online remote simulation-based assessment, its psychometric properties and students’ feedback towards this new learning and assessment tool opportunity. A virtual web-based simulation examination consisting of three infectious disease scenarios was administered in an undergraduate pharmacy course to assess physically distanced students for medium-level clinical pharmacy CSs. Students responded through written typed communication to online enquiries from health professionals, patients or caretakers’ avatars. Rasch analysis and a feedback survey were employed to measure the reliability of online simulation and to understand students’ experiences with the new web-based tool. This study found that the remote simulation examination had good reliability (ir = 0.95, pr = 0.73) for measuring medium-level clinical pharmacy CSs. The students’ survey indicates that the web-based simulation activities improved knowledge, engagement and cognitive reasoning. This article contributes to the pharmacy education literature by illustrating how a different type of assessment is feasible and reliable for evaluating students’ CSs for final examinations and potentially has equitable opportunities for distance-learning students to enhance their learning capabilities and show their performances. Future research to investigate the potential of remote online simulation examinations for inter-professional learning and assessment is highly recommended.