Serious game evaluation as a meta‐game

Abstract

Purpose -This paper aims to briefly outline the seamless evaluation approach and its application during an evaluation of ORIENT, a serious game aimed at young adults. Design/methodology/approach -In this paper, the authors detail a unobtrusive, embedded evaluation approach that occurs within the game context, adding value and entertainment to the player experience whilst accumulating useful data for the development team. Findings -The key result from this study was that during the "seamless evaluation" approach, users were unaware that they had been participating in an evaluation, with instruments enhancing rather than detracting from the in-role game experience. Practical implications -This approach, seamless evaluation, was devised in response to player expectations, perspectives and requirements, recognising that in the evaluation of games the whole process of interaction including its evaluation must be enjoyable and fun for the user. Originality/value -Through using seamless evaluation, the authors created an evaluation completely embedded within the "magic circle" of an in-game experience that added value to the user experience whilst also yielding relevant results for the development team.

Five Degrees of Happiness

“…This decision reflects the approach used in activity books for children (e.g. annuals, summer comic specials) where a range of activities and formats are used to maintain interest and the findings of [14] where diversity in instrument aesthetic was identified as critical in not boring users during the evaluation. For example, in 'New People, New Places,' an alternative numeric scale is used, see figure 11.…”

Engaging Children in Interactive Application Evaluation

“…Quantitative measures to assess the user's engagement, interaction and immediate learning were based on questionnaires developed for assessing the user experience in VLEs populated by embodied characters, based on Hall et al, (2013) and Hall, Woods and Aylett, (2006). Theory of Mind questions required by the R&D team used to assess children's advice to Tom are embedded in the conversation the child has with Tom, following the approach in (Hall, Woods, & Hall, 2009).…”

“…Our approach to data collection transformation has had a significant impact on the user's perception of what they are doing. Users are usually unaware that they are completing questionnaires, being assessed on their learning or participating in a focus group for example, as the props that they are engaging with are embedded and just part of their in-role experience (Hall et al, 2013). We recognize that to any experienced evaluator they are clearly questionnaires and focus groups, however, this is not the user perception, with the instruments…”

Engaging Children in Interactive Application Evaluation

“…Our approach to data collection transformation has had a significant impact on the user's perception of what they are doing. Users are usually unaware that they are completing questionnaires, being assessed on their learning or participating in a focus group for example, as the props that they are engaging with are embedded and just part of their in-role experience (Hall et al, 2013). We recognize that to any experienced evaluator they are clearly questionnaires and focus groups, however, this is not the user perception, with the instruments The MESSY was divided into three separate sets of questions:The Epic Quiz -children identify on a scale how similar/dissimilar they are to the items.…”