Proof of concept for a multimethod approach to evaluate the usability of websites with public health content: The case of Alzheimer association websites.

Abstract

Methods for assessing usability are advancing rapidly and include think-aloud protocols, objective measures such as task timing, and ultrasensitive measures such as eye tracking and page recording. In this study, we provide proof-of-concept for the new, combined use of these three types of measures within a quasi-experimental paradigm focused on a question relevant to public health. The quasi-experiment specifically involved assessing the usability of four English-language Alzheimer websites from around the world (USA, U.K., Ireland, Canada), all of which contained information about cognitive health. Participants completed two tasks; one requiring them to identify strategies to prevent dementia and another requiring them to locate a definition of Alzheimer's disease. We hypothesized that the websites would have superior usability for the former task and inferior usability for the latter task. We also hypothesized positive correlations among eye tracking measures, simpler performance measures (e.g., task completion time), and subjective usability measures. We found that the new quasiexperimental protocol facilitated comparison of tasks and websites. It also facilitated interpretation by permitting comparison across measures. Overall, this study provides proof of concept for the use of this multimethod approach to evaluate the usability of websites. It also provides information that could potentially be used to advance further pilot or experimental hypothesis testing on this topic. Public Significance StatementIn the "information age," the internet is an important venue for translating health information to the public. Unfortunately, previous research has shown that some of the most accurate and informative websites are ultimately not very usable. After reviewing the state of usability science, we noticed that while some researchers rely on subjective measures (how usable do you think the website was?), others rely on simple performance measures like how long it took to find relevant information, and still others use eye tracking software to document exactly how the user navigates the website. In this study, we designed a protocol that combines each type of measure to answer a public health research question. The study provides evidence that these measures are comparable and that using them together can facilitate interpretation.