Abstract. Evaluating the usability of mobile systems raises new concerns and questions, challenging methods for both lab and field evaluations. A recent literature study showed that most mobile HCI research projects apply lab-based evaluations. Nevertheless, several researchers argue in favour of field evaluations as mobile systems are highly context-dependent. However, field-based usability studies are difficult to conduct, time consuming and the added value is unknown. Contributing to this discussion, this paper compares the results produced by a laboratory-and a field-based evaluation of the same context-aware mobile system on their ability to identify usability problems. Six test subjects used the mobile system in a laboratory while another six used the system in the field. The results show that the added value of conducting usability evaluations in the field is very little and that recreating central aspects of the use context in a laboratory setting enables the identification of the same usability problem list.

Coolness has recently started to be explored as a design goal for interactive products from practitioners as well as researchers within HCI, but there is still a need to further operationalize the concept and explore how we can measure it. Our contribution in this paper is the COOL Questionnaire. We based the creation of the questionnaire on literature suggesting that perceived coolness is decomposed to outer cool (the style of a product) and inner cool (the personality characteristics assigned to it). In this paper, we focused on inner cool and we identified eleven inner cool characteristics. These were used to create an initial pool of question items and 2236 participants were asked to assess 16 mobile devices. By performing exploratory and confirmatory factor analyses we identified three factors that can measure the perceived inner coolness of interactive products: Desirability, rebelliousness, and usability. These factors and their underlying 16 question items comprise the COOL questionnaire. The whole process of creating the questionnaire is presented in detail in this paper and we conclude by discussing our work against related work on coolness and human computer interaction.

Global Positioning System (GPS) navigation systems were amongst the top selling consumer technologies in 2008 and research has indicated that such technologies could affect driving behaviour. In this paper, we study how different output configurations (audio, visual and audio-visual) of a GPS system affect driving behaviour and performance. We conducted field experiments in real traffic with 30 subjects. Our results illustrated that visual output not only causes a substantial amount of eye glances, but also led to a decrease in driving performance. Adding audio output decreased the number of eye glances, but we found no significant effects on driving performance. Although the audio configuration implied much fewer eye glances and improved driving performance, several participants expressed preference for the audio/visual output.

Research methods have been objects of discussions for decades and defining research methods is still a quite considerable challenge. However, it is important to understand research methods in different disciplines as it informs us on future directions and influences on the discipline.We conduct a survey of research methods in paper publications. 105 papers on children's technology design are classified on a two-dimensional matrix on research method and purpose. Our results show a strong focus on engineering of products as applied research and on evaluation of developed products in the field or in the lab. Also, we find that much research is conducted in natural setting environments with strong focus on field studies.

Recently within HCI, design approaches have appeared, which deviate from traditional ones. Among them critical design introduces deliberate provocations in order to challenge established perceptions and practices. We have engaged ourselves with this design approach out of interest in understanding how to use provocation in research through design. Towards this end, we report on a field study with four families that used an aesthetically, functionally and conceptually provocative future probe. The purpose of the probe was to challenge existing energy consuming practices through provocation and make its users reflect on them. The paper describes how all three provocative aspects were addressed, and our findings demonstrate how they were experienced in the real world, and how they impacted our research through design approach. We conclude by presenting reflections on how to design provocations, and reflections on the impact of provocations for research through design in general.