A Detailed Rubric for Assessing the Quality of Teacher Resource Apps

Abstract

Since the advent of the iPhone and rise of mobile technologies, educational apps represent one of the fastest growing markets, and both the mobile technology and educational app markets are predicted to continue experiencing growth into the foreseeable future. The irony, however, is that even with a booming market for educational apps, very little research regarding the quality of them has been conducted. Though some instruments have been developed to evaluate apps geared towards student learning, no such instrument has been created for teacher resource apps, which are designed to assist teachers in completing common tasks (e.g., taking attendance, communicating with parents, monitoring student learning and behavior, etc.). Moreover, when teachers visit the App Store or Google Play to learn about apps, the only ratings provided to them are generic, five-point evaluations, which do not provide qualifiers that explain why an app earned three, two, or five points. To address that gap, previously conducted research related to designing instructional technologies coupled with best practices for supporting teachers were first identified. That information was then used to construct a comprehensive rubric for assessing teacher re-source apps. In this article, a discussion that explains the need for such a rubric is offered before describing the process used to create it. The article then presents the rubric and discusses its different components and potential limitations and concludes with suggestions for future research based on the rubric.

A Comprehensive Rubric for Evaluating EduVR

“…These aspects negate the reliability and validity of app stores' app ratings, which will be next explained. Cherner et al (2016) explained that the starred rating systems employed by app stores are fundamentally flawed due to their subjective nature:…”

“…Why an app might score a "two" to one person may well be the reason another person scored it as a "three" or higher, which complicates the type of information that can be learned from the ratings. (p. 119) Cherner et al (2016) explained that these star scores do not provide a methodical or criterion-referenced evaluation of an app's content and quality, which results in a need to develop criterion-referenced instruments to help ensure consistent, reliable, and valid evaluations. While the personal subjectivities of evaluators are acknowledged as a limiting characteristic of rubrics as well (Cherner et al, 2016), the specific language of rubric indicators provides guidance for evaluators to select their scores consistently and accurately (Crusan, 2010).…”

“…Buckler (2012) created another rubric for assessing apps to support adult learners with special needs. These early rubrics were critiqued due to being underdeveloped, not utilizing prior research, and lacking applicability (Cherner et al, 2016;Lee & Cherner, 2015). To explain, both rubrics used general terms and unclear criteria to make evaluative decisions about the quality of apps.…”

“…By the early 2010s, schools across developed countries were launching 1:1 initiatives that supplied all students and teachers with a mobile device to use, and developers were designing and releasing apps at a rapid rate. During this time, Lee and Cherner (2015) and Cherner et al (2016) published their comprehensive rubrics for evaluating educational apps. Lee and Cherner's (2015) instructional app rubric consisted of 24 dimensions that assessed apps based on their instructional features, design elements, and engaging qualities.…”

“…Though trailblazing when each of these rubrics were released (Buckler, 2012;Cherner et al, 2016;Lee & Cherner, 2015;Reeves & Harmon, 1993;Walker, 2010), advances in technology, deeper understandings of the learning sciences, and subjectivities embedded within their dimensions have resulted in the need to update their work. The natural progression of technology along with the growing interest, investment, and availability of virtual reality in the market has again established a need for a new rubric designed specifically for evaluating eduVR.…”