Evaluating the usefulness of mobile services based on captured usage data from longitudinal field trials

Abstract

It is widely discussed whether the usefulness of mobile systems and services should be evaluated in the field or in the lab. The aim of this paper is to make a case for conducting longitudinal field trials when evaluating the usefulness of mobile services, and to partly base such evaluations on quantitative usage data automatically captured on the device. A framework for automatic capture and analysis of such data is introduced, and it is shown how it has been used to evaluate a mobile service for diabetes management. Results and experiences based on a three month field trial with a single diabetic user is presented and it is demonstrated how these results can be used to make inferences about the use of the service with regard to functionality and usage patterns.

Mobile Phone-Based Self-Management Tools for Type 2 Diabetes: The Few Touch Application

“…The underlying premise is that self-help tools must be as automatic and easy to use as possible to avoid adding to the burdens of people with chronic diseases, as argued by Jensen and Larsen. 13 The chosen elements of the Few Touch application are based on the cornerstones of diabetes management: healthy diet, blood glucose (BG) management, and physical activity. The designed elements are integrated and interconnected in a mobile terminal-based application.…”

The analysis of contextual factors on the use of smartphones applications

“…According to Mallick [7], the experiences show that human beings usually interact with systems in unusual ways. Thus, user interaction tests conducted in real scenarios are essential to delineate the users' preferences and the consequent adaptation of products addressed to them [5].…”

“…To conduct the studies with such a broad reach, it is necessary to use methodologies and techniques that allow to carrying out experiments that are able to collect data contextualized with the scenarios where the interactions take place [4]. This fact provokes several discussions regarding the place where experiments are conducted (in the field or in the laboratory) [5], as well as the techniques…”

Evaluation of the influence of contextual factors on the interactions with applications for smartphones

“…Uma vez que a previsão de tais formas de interação não é uma tarefa trivial, a realização de testes constitui uma atividade crítica para a garantia da usabilidade do aplicativo associado ao contexto de uso considerado. Desta forma, a inclusão de usuários e cenários reais nos testes é imprescindível para o delineamento das preferências dos usuários e a consequente adequação dos produtos a eles destinados [5].…”

“…Para a realização de estudos com esta abrangência, são necessárias metodologias e técnicas que permitam realizar experimentos com a capacidade de coletar dados contextualizados aos cenários onde as interações ocorrem [4]. Esse fato desencadeia diversas discussões quanto ao local da realização dos experimentos (em campo ou em laboratório) [5], bem como as técnicas que podem ser utilizadas para a extração do melhor conjunto de dados que caracterizem os experimentos [6].…”