A framework for mobile evaluation

Abstract

In this paper we present a software framework which supports the construction and evaluation of mixedfidelity prototypes for mobile devices. The framework is available for desktop and mobile devices and allows designers and users to 1) test the prototypes on actual devices; 2) gather usage information, both passively and actively supporting contextual and ubiquitous evaluation; 3) convey common prototyping procedures with effective data gathering methods that can be used on ubiquitous scenarios; 4) support in-situ prototyping and participatory design on-the-go. We address the framework's features and its contributions to the evaluation of applications for mobile devices and the field of mobile interaction design, presenting real-life case studies and achieved results.

Sampling and Reconstructing User Experience

“…We will begin this review with two projects which have developed tools to integrated aspects of both easily creating ubicomp prototypes as well as evaluating them (Carter et al, 2007, de Sá et al, 2008.…”

Adapting Evaluation to Study Behaviour in Context

“…While the authors identify the challenges of long-term studies, no strategies are offered to overcome them. Iterative participatory design practices for developing the usability of mobile devices offer to some extent more 'agile' evaluation techniques (de Sá et al, 2008), and there are some existing demonstrations of remote, in situ data collection systems (Carter et al, 2007;Consolvo & Walker, 2003;Froehlich et al, 2007). Froehlich et al (2007) have explored context-dependent 'experience sampling' systems that prompt the user for explanatory input when a mobile device detects that it is in a context of interest.…”

System and Context – On a Discernable Source of Emergent Game Play and the Process-Oriented Method

“…We see this difference between system and context as the source of emergent game play, which have to be taken into account methodologically when it comes to study play experiences and to evaluate a mobile game. Today researchers deploy traditional HCI methods and seek to adapt them to the novel situation [4]. Ethnographers around Crabtree and the Mixed Reality Lab Nottingham conceive game play as the result of players' actions and consequently look for the particular methods players employ to organize their play action within their own context [5], [6].…”