This paper investigates a novel methodological approach in performing user evaluation trials for pervasive games. The evaluation process of a pervasive role-playing game, Barbarossa, is used as a case study. Barbarossa involves a preliminary and a main execution phase. The former is freely available to anyone and may be played anytime/anywhere requiring no organizational and orchestration investments from the investigators team. The latter defines three interdependent player roles acted by players who need to collaborate in a treasure hunting game to achieve a common game goal. The eligibility of players for participating in the main game phase derives from those ranked relatively high in the preparatory phase. Drawing on concepts of cultural theory, we design the preparatory phase as an affective environment out of which the potential evaluators will emerge. The main hypothesis investigated is that the execution of such cost-effective preparatory phases may serve as a means for recruiting highly qualified subjects for user trials on pervasive game research prototypes, thereby increasing the reliability and quality of evaluation results. This hypothesis has been validated through the user evaluation trials performed on both the Barbarossa game phases.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.