Usability Assessment of the Missouri Cancer Registry’s Published Interactive Mapping Reports: Round One

Abstract

Background Many users of spatial data have difficulty interpreting information in health-related spatial reports. The Missouri Cancer Registry and Research Center (MCR-ARC) has produced interactive reports for several years. These reports have never been tested for usability.Objective The aims of this study were to: (1) conduct a multi-approach usability testing study to understand ease of use (user friendliness) and user satisfaction; and (2) evaluate the usability of MCR-ARC’s published InstantAtlas reports.Methods An institutional review board (IRB) approved mixed methodology usability testing study using a convenience sample of health professionals. A recruiting email was sent to faculty in the Master of Public Health program and to faculty and staff in the Department of Health Management and Informatics at the University of Missouri-Columbia. The study included 7 participants. The test included a pretest questionnaire, a multi-task usability test, and the System Usability Scale (SUS). Also, the researchers collected participants’ comments about the tested maps immediately after every trial. Software was used to record the computer screen during the trial and the participants’ spoken comments. Several performance and usability metrics were measured to evaluate the usability of MCR-ARC’s published mapping reports.ResultsOf the 10 assigned tasks, 6 reached a 100% completion success rate, and this outcome was relative to the complexity of the tasks. The simple tasks were handled more efficiently than the complicated tasks. The SUS score ranged between 20-100 points, with an average of 62.7 points and a median of 50.5 points. The tested maps’ effectiveness outcomes were better than the efficiency and satisfaction outcomes. There was a statistically significant relationship between the subjects’ performance on the study test and the users’ previous experience with geographic information system (GIS) tools (P=.03). There were no statistically significant relationships between users’ performance and satisfaction and their education level, work type, or previous experience in health care (P>.05). There were strong positive correlations between the three measured usability elements.ConclusionsThe tested maps should undergo an extensive refining and updating to overcome all the discovered usability issues and meet the perspectives and needs of the tested maps’ potential users. The study results might convey the perspectives of academic health professionals toward GIS health data. We need to conduct a second-round usability study with public health practitioners and cancer professionals who use GIS tools on a routine basis. Usability testing should be conducted before and after releasing MCR-ARC’s maps in the future.

Usability Assessment of the Missouri Cancer Registry’s Published Interactive Mapping Reports: Round Two

“…Figure (4) shows that the highest ORE rates were for the tasks #1, 2, 3, 9& 10 and they were categorized as easy tasks except the task #3, which is ranked as a complex task. Task #8 got an OTE of 93% in comparison to almost OTE of 90% in the first round [17]. Task #6, a ranked complicated task, followed with an OTE of 88% ORE rate, in comparison to the OTE of about 95% in the first round [17].…”

“…The investigators selected a mixed methodology tactic. The tested mapping reports are already published on the MCR-ARC (see Multimedia Appendices 1 and 2) [17][18][19]. The trial elements were the same Round One's components: A pretest questionnaire, the multi-task usability test, and the System Usability Scale (SUS) per every participant respectively [17,20].…”

“…In the first round of the study, the investigators conducted a usability testing trial to assess the usability of the two published Missouri Cancer Registry and Research Center (MCR-ARC)'s interactive reports. The first round's participants were faculty and staff of Master of Public Health OJPHI program (MPH) and the Department of Health Management and Informatics (HMI) at University of Missouri-Columbia [17].…”

“…In the current study, we conducted a second round of the same first round's methodology to measure effectiveness and efficiency of the published InstantAtlas reports of MCR-ARC after we modified them according to the first round results [17]. In this round, we selected a convenience sample of cancer professionals who attended the North America Association of Central Cancer Registries (NAACCR) conference-June, 2016.…”

“…Some of these tasks are multi-stepping tasks and needs special knowledge to be figured out. From the first round of the study, the same tasks in addition to the task # 3 got the lowest completion rates [17].…”