An N-of-1 Evaluation Framework for Behaviour Change Applications

Abstract

Mobile behaviour change applications should be evaluated for their effectiveness in promoting the intended behavior changes. In this paper we argue that the "gold standard" form of effectiveness evaluation, the randomised controlled trial, has shortcomings when applied to mobile applications. We propose that N-of-1 (also known as single case design) based approaches have advantages. There is currently a lack of guidance for researchers and developers on how to take this approach. We present a framework encompassing three phases and two related checklists for performing N-of-1 evaluations. We also present our analysis of using this framework in the development and deployment of an app that encourages people to walk more. Our key findings are that there are challenges in designing engaging apps that automate N-of-1 procedures, and that there are challenges in collecting sufficient data of good quality. Further research should address these challenges.

Antifragile Behavior Change Through Digital Health Behavior Change Interventions

“…It also aligns with the conceptual and technological architecture proposed by the Behavioral Intervention Technology Model [ 27 ]. Finally, it follows McCallum et al’s [ 28 ] “n-of-1 evaluation framework for behavior change applications” to inform which user variables are collected, aiming to gain information that can be aggregated and used to improve tailoring with high internal, external, and social validity [ 28 ]. Rather than introducing new features, our aim is to demonstrate how our design principles might be implemented in a DHBCI similar to existing ones by highlighting or modifying features with demonstrated real-world effectiveness.…”