User experience evaluation in the wild

Jambon, Francis; Meillon, Brigitte

doi:10.1145/1520340.1520619

Cited by 20 publications

(11 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, constructing an accurate understanding of what takes place during the experiment is a major challenge when conducting studies in the wild. The issues one encounters include users anticipating and modifying their behavior to satisfy the researcher's needs [5] [17], discrepancies between self-reflection [11] and anticipated needs [23] and the logged use of the system, and analyzing the large scale, heterogeneous data captured during the study [15]. Clearly, no single evaluation method can account for all of these concerns, and researchers have to choose from and adapt existing approaches to fit the unique needs of their studies.…”

Section: Related Workmentioning

confidence: 99%

“…Experimenting in real-world situations, while technologically and methodologically demanding, can inform us about the real usage of the new technologies [15], especially when attempting to understand the context-dependent factors such as mobility and the effects of environment [17]. However, a key challenge in conducting these evaluations is that different assessment goals can make it difficult to select the appropriate evaluation method [2].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluating the experiential user experience of public display applications in the wild

Keskinen

Hakulinen

Heimonen

et al. 2013

Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia

View full text Add to dashboard Cite

Studying pervasive systems in the wild has recently gained significant interest. However, few methods exist that focus on the subjective of user experience of such systems rather than objective metrics, like performance and task success. Especially multimodal interaction in this context poses challenges to understanding how different input and output methods affect the users' experience. We present a new method for evaluating the experiential user experience of interactive systems. It combines two existing approaches from different fields: a questionnairebased evaluation method called SUXES, intended for evaluating user expectations and experiences, and a theoretical experience framework, Experience Pyramid, originally developed for analyzing and improving experiential tourism products. The new method was used in two field studies of multimodal public display applications. Our findings show that the method is a practical approach for user experience evaluation in the wild, especially in the case of pervasive applications that aim to provide novel experiences rather than facilitate task-oriented information access.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Evaluating the experiential user experience of public display applications in the wild

Keskinen

Hakulinen

Heimonen

et al. 2013

Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia

View full text Add to dashboard Cite

show abstract

“…In another example, Jambon and Meillon [27] equipped a group of skiers with a self-performance system, which relayed usage data from a camera, accelerometer and GPS back to the researchers via wireless Internet.…”

Section: Lab Studiesmentioning

confidence: 99%

“…That is perhaps also why the labels "in-situ" and "in-the-wild" have been adapted by some papers (e.g. [8,12,24,27,54,55,63]) as they are really much better at capturing the essence of what field studies should be about. So, just like a lab study without control and replicability would be considered a poor one, a field study that does not really take the researcher into an uncontrolled real world situation is perhaps not a good one either.…”

Section: Beyond Non-wild Snap-shot Field Studiesmentioning

confidence: 99%

Was it worth the hassle?

Kjeldskov

Skov

2014

Proceedings of the 16th International Conference on Human-Computer Interaction With Mobile Devices &Amp; Services

View full text Add to dashboard Cite

Evaluation is considered one of the major cornerstones of human-computer interaction (HCI). During the last decade, several studies have discussed pros and cons of lab and field evaluations. Based on these discussions, we conduct a review to explore the past decade of mobile HCI research on field and lab evaluation, investigating responses in the literature to the "is it worth the hassle?" paper from 2004. We find that while our knowledge and experience with both lab and field studies have grown considerably, there is still no definite answer to the lab versus field question. In response we suggest that the real question is not if -but when and how -to go into the field. In response we suggest moving beyond usability evaluations, and to engage with field studies that are truly in-the-wild, and longitudinal.

show abstract

“…However, the observer's presence can disturb the users and a human observer can miss quite a lot of what is really happening. Logging and recording, to recreate the experience later on can be used as in [8]. Those methods give a perfect but partial rendering of the experience, and they still require a long time to be analyzed afterwards, and much equipment and organization during the evaluation.…”

Section: State Of the Art / Previous Workmentioning

confidence: 99%

A real-world study of an audio-tactile tourist guide

Szymczak

Rassmus-Gröhn

Magnusson

et al. 2012

Proceedings of the 14th International Conference on Human-Computer Interaction With Mobile Devices and Services

View full text Add to dashboard Cite

This paper reports on the in-context evaluation of an audiotactile interactive tourist guide -one test was done in a medieval city center, and the other was done at an archaeological site. The activity theory framework was used as a perspective to guide design, field-study and analysis. The evaluation shows that the guide allows users to experience an augmented reality, while keeping the environment in focus (in contrast with the common key-hole like experience that onscreen augmented reality generates). The evaluation also confirms the usefulness of extending the vibrational feedback to convey also distance information as well as directional information.

show abstract

User experience evaluation in the wild

Cited by 20 publications

References 5 publications

Evaluating the experiential user experience of public display applications in the wild

Evaluating the experiential user experience of public display applications in the wild

Was it worth the hassle?

A real-world study of an audio-tactile tourist guide

Contact Info

Product

Resources

About