Usability and Workflow Evaluation of “RhEumAtic Disease activitY” (READY)

Abstract

SummaryBackground: RhEumAtic Disease activitY (READY) is a mobile health (mHealth) application that aims to create a shared platform integrating data from both patients and physicians, with a particular emphasis on arthritis disease activity. Methods: We made READY available on an iPad and pilot implemented it at a rheumatology outpatient clinic. We conducted 1) a usability evaluation study to explore patients' and physicians' interactions with READY, and 2) a time motion study (TMS) to observe the clinical workflow before and after the implementation. Results: A total of 33 patients and 15 physicians participated in the usability evaluation. We found usability problems in navigation, data entry, pain assessment, documentation, and instructions along with error messages. Despite these issues, 25 (75,76%) patients reported they liked READY. Physicians provided mixed feedback because they were concerned about the impact of READY on clinical workflow. Six physicians participated in the TMS. We observed 47 patient visits (44.72 hours) in the pre-implementation phase, and 42 patient visits (37.82 hours) in the post-implementation phase. We found that patients spent more time on READY than paper (4.39mins vs. 2.26mins), but overall, READY did not delay the workflow (pre = 52.08 mins vs. post = 45.46 mins). This time difference may be compensated with READY eliminating a workflow step for the staff. Conclusion: Patients preferred READY to paper documents. Many found it easier to input information because of the larger font size and the ease of 'tapping' rather than writing-out or circling answers. Even though patients spent more time on READY than using paper documents, the longer usage of READY was mainly due to when troubleshooting was needed. Most patients did not have problems after receiving initial support from the staff. This study not only enabled improvements to the software but also serves as good reference for other researchers or institutional decision makers who are interested in implementing such a technology.

Methodological Quality of Manuscripts Reporting on the Usability of Mobile Applications for Pain Assessment and Management: A Systematic Review

“…Besides, most manuscripts did not report on the measurement of effectiveness, efficiency, and satisfaction, because they fail to use a combination of methods that allows assessing these three domains [27]. Only eight out of the 31 manuscripts included [19,29,32,37,38,[54][55][56] assessed all the three domains of usability, despite existing recommendations to include measures of efficiency, effectiveness, and user satisfaction since a narrower selection of usability measures may lead to unreliable conclusions about the overall usability of the app [64]. Furthermore, there was an inconsistency between what was reported in the methodology section and the results presented.…”

“…All manuscripts except three [34][35][36] assessed the domain "satisfaction". Only eight manuscripts described methods compatible with the assessment of all the three domains of usability (efficiency, effectiveness, and satisfaction), combining objective and subjective indicators, such as interviews, task completion rates and measuring the time participants needed to complete each task [19,29,32,37,38]. However, the results are not clearly and adequately reported in some manuscripts.…”

“…Regarding the involvement of end-users in usability assessment, 23 manuscripts assessed apps that were intended for individuals with specific pathologies and used a sample of individuals with characteristics similar to the target group of end-users for usability assessment [19,29,[31][32][33][34][36][37][38][40][41][42][43][44][45][46][47][48][49][50][51][52][53]. Participants in these studies included, among others, individuals with fibromyalgia and other rheumatological diseases [37,38,40,43,44], and individuals with headaches [29,32,33,[45][46][47][48], cancer-related pain [36,[49][50][51], or chronic pain in general [19,41,52,53], individuals with Down syndrome [34] and individuals who are wheelchair users [31,42]. Of these 23 studies, five al...…”

Assessing Rheumatoid Arthritis Disease Activity With Patient‐Reported Outcomes Measurement Information System Measures Using Digital Technology

“…For physicians, these findings can inform their implementation of PROs into routine clinic settings where time spent collecting patient data must be kept to a minimum to avoid meaningfully impacting office workflow. Some electronic systems are currently available for rheumatology office settings to collect PROs (including PROMIS CATs) in a highly streamlined fashion to capitalize on these efficiencies .…”

Using Health Information Technology to Support Use of Patient-Reported Outcomes in Rheumatology

“…READY (RhEumAtic Disease activitY) is an app (iPad)-based stand-alone ePRO application that enables the PRO data capture process to occur at physician offices using a tablet. 6 It uses multiple validated PRO questionnaires that reflect symptoms and quality of life of rheumatic diseases, with the option to show trended scores. Usability issues were found in READY, including touchscreen sensitivity (not sensitive enough when users attempted to complete an action), interface design (layout and font size), and instruction and error messages.…”