A comparative study between a computer-based and a mobile-based assessment

Abstract

Purpose The purpose of this study is to compare the overall usability and user experience of desktop computers and mobile-devices when used in a summative assessment in the context of a higher education course. Design/methodology/approach The study follows a between-groups design. The participants were 110 first-year undergraduate students from a European university. Students in the experimental group participated in the assessment using mobile devices, whereas students in the control group participated using desktop computers. After the assessment, students self-reported their experiences with computer-based assessment (CBA) and mobile-based assessment (MBA), respectively. The instruments used were the user experience questionnaire and the system usability scale. Findings Attractiveness and novelty were reported significantly higher in the experimental group (MBA), while no significant differences were found between the two groups in terms of efficiency, perspicuity, dependability and stimulation. The overall score for the system usability was not found to differ between the two conditions. Practical implications The usability and user experience issues discussed in this study can inform educators and policymakers about the potential of using mobile devices in online assessment practices, as an alternative to desktop computers. Originality/value The study is novel, in that it provides quantitative evidence for the usability and user experience of both desktop computers and mobile devices when used in a summative assessment in the context of a higher education course. Study findings can contribute towards the interchangeable usage of desktop computers and mobile devices in assessment practices in higher education.

Determinants of high school students’ digital reading flow experience: an experimental study

“…A seven-week online discussion experiment showed that smartphone use outperforms computer use in learning preparation, learning motivation, learner-interface interaction and flow experience (Kyung, 2015). A recent study found that smartphone users report higher attractiveness and novelty than computer users when engaging in summative assessments (Nikou and Economides, 2019). Therefore, this study proposes the following hypothesis:…”

Use of an e-toolkit in the development of digital competencies in Weeks of International Teaching

“…The first was a Likert-scale questionnaire to survey the participants' views of the workshops. The questionnaire was based on the work of Nikou & Economides (2019). However, quantitative methods can give us a clear understanding of the findings and the reason behind them, i.e.…”

Virtual learner experience (VLX)

“…This includes the user's physical, cognitive, and emotional experience. The factors that are considered in measuring this experience vary in the literature and identifying these factors is an ongoing process in various studies (International Organization for Standardization, 2018; Moczarny et al, 2012;Nikou & Economides, 2019;Rhiu et al, 2020;Rogers et al, 2002). However, certain design aspects, like authenticity, immersion, and instructional support, that influenced the virtual learner experience (VLX) while using the VRLE have not been considered as part of UX in the current literature, and do not form part of the UX of general computer systems.…”