An evaluation tool for research of user behavior in a realistic mobile environment

Abstract

User behavior is significantly influenced by the surrounding environment. Especially complex and dynamically changing environments (like mobile environment) are represented by a wide variety of extraneous variables, which influence the user behavior in an unpredictable and mostly uncontrolled way. For researchers it is challenging to measure and analyze the user behavior in such environments. We introduce a complex tool -the IVE tool -which provides a unique way of context visualization and synchronization of measured data of various kinds. Thanks to this tool it is possible to efficiently evaluate data acquired during complex usability tests in a mobile environment. The functionality of this tool is demonstrated on the use-case "Navigation of visually impaired users in the building with support of a navigation system called NaviTerier". During the experiment we focused on collection and analysis of data that may show user stress and which may influence his/her ability to navigate. We analyzed objective data like Galvanic Skin Response parameter (GSR), Heart Rate Variability parameters (HRV) and audio video recordings and also subjective data like the user's subjective stress feeling and observation of the user's behavior.

Was it worth the hassle?

“…Similarly, Lumsden et al [45] simulate a city street using a surround sound system in the lab to deliver recorded city street noise, and projections on the floor to create virtual obstacles that the user should try to avoid when walking around the lab. Such simulation of mobility is also found in other lab studies, for example in the work by Wilson et al [62] and Maly et al [48] where test subjects were asked to walk and navigate a track with a number of obstacles, or in Banard et al [6] where they are asked to walk on a treadmill. An example of including the presence and activity of other people in a lab study is found in the work of Leitner et al [44] who simulate a traffic accident to users of an emergency response system.…”

Challenges for usability testing in ubiquitous systems

“…However, most papers performed usability evaluation in Smart Homes [2,10,11,12,13] and Navigating System [14,15,16,17,18]. Other work performed usability evaluation in other domains [7,9,19,20,21,22] or not really performed any evaluation [8,23,24,25].…”

“…Most papers [2,7,9,10,12,13,14,15,16,17,20,21,22] performed usability testing in controlled environments, not in real environments. Moreover, few papers presented specific approaches to usability evaluation for ubiquitous applications and, these approaches that have been identified are not specific to usability tests.…”

Capturing community in mobility: Mobile methods for community informatics

“…In response to these issues, researchers use simulations of realistic and natural environments in lab studies. Maly, Mikovec, Vystrcil, Franc, and Slavik (2013), for example, describe a lab experiment to study the influence of stress on the uses of a mobile navigation system for people with visual impairment. In order to simulate a stressful environment, researchers set up auditory and haptic stimuli as well as other barriers on the navigation route in the lab.…”