Enhancing Learning Management Systems Utility for Blind Students: A Task-oriented, User-Centered, Multi-Method Evaluation Technique

Abstract

Executive SummaryThis paper presents a novel task-oriented, user-centered, multi-method evaluation (TUME) technique and shows how it is useful in providing a more complete, practical and solution-oriented assessment of the accessibility and usability of Learning Management Systems (LMS) for blind and visually impaired (BVI) students. Novel components of TUME include a purposeful integration of a multi-theoretic foundation and multiple methods to accurately identify users' accessibility and usability problems in Web interaction and identify design problems and solutions to ensure technical feasibility of recommendations. The problems identified by TUME remain hidden from extant evaluation methods -therefore, these problems remain in Web-based applications. As a result, evaluation of Web-based applications remains confounded by users' Web interaction challenges; their utility for specific user types remains unclear. Without appropriate evaluation of users' problems and challenges in using Web-based applications, we cannot begin to solve these problems and challenges. This paper demonstrates how TUME can be used to identify the unique problems and challenges of specific user types in using Web-based applications and suggests potential solutions. The outcome is an accurate understanding of specific design elements that present roadblocks and challenges for the user in interacting with the Web-based application and feasible design modifications to potentially improve the utility of these applications for specific user types.We illustrate the application of TUME to test the utility of LMS for BVI students. We use a popular LMS and an on-line exam as a representative task. We present results from three assessments that are essential components of TUME. Assessment I uses verbal protocol analysis to identify where and why BVI participants face LMS interaction problems in completing an exam. Assessment II performs text analysis of accessibility and usability design standards to identify design errors in the LMS exam environment. AsMaterial published as part of this publication, either on-line or in print, is copyrighted by the Informing Science Institute. Permission to make digital or paper copy of part or all of these works for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage AND that copies 1) bear this notice in full and 2) give the full citation on the first page. It is permissible to abstract these works so long as credit is given. To copy in all other cases or to republish or to post on a server or to redistribute to lists requires specific permission and payment of a fee. Contact Publisher@InformingScience.org to request redistribution permission. Editor: Anthony ScimeEnhancing Learning Management Systems Utility for Blind Students sessment III employs recursive abstraction analysis of Web developers' comments to understand why these problems occur and how they may be addressed feasibly. Together, these assessments show how TUME...

Network Project As A Means Of Implementing School And University Project Activities

“…The analysis of home and foreign sources on project-based learning enabled us to highlight its following features: correlation with objective reality, correlation with the subject of the activity; methodological relevance of the project; focus on the end-product (Babu, 2013;Devkota, 2017;Förster, 2011;Mairinger, 2017;Schüßler, 2004;Tabor, 2013).…”

Heuristic evaluations of the accessibility of learning management systems (LMSs) as authoring tools for teachers

“…The authors evaluated the Merlin LMS used at the University of Zagreb, and defined the guidelines for designing accessible e-learning systems covering aspects such as Web site architecture, navigation and design, descriptive tags, keyboard shortcuts and auditory feedback. An approach for evaluating the accessibility, usability and utility of LMSs for blind and visually impaired students was proposed by Babu and Singh (2013) based on task-oriented user-centered multi-method evaluation (TUME) technique. Accessibility evaluation tools have also been developed.…”

Haze in the digital library: design issues hampering accessibility for blind users

“…They comprise an atypical population that interacts with digital resources by listening to information verbalized by the SR. The sight-centred design of digital resources presents challenges in SR-mediated interaction (Babu and Singh, 2013). DLs are highly complex by nature, as content is heterogeneous in format and system dimensions (O'Day and Nardi, 2003).…”