Multimodal mobile interactions

Williamson, Julie R.; Crossan, Andrew; Brewster, Stephen

doi:10.1145/2070481.2070551

Cited by 39 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in face-to-face communication scenarios using situated visualization [35], it can be essential to consider the potential for misunderstandings and social awkwardness arising from hand-pointing interactions directed at others. In addition, in various public settings where users independently perform tasks like crossing the street or taking the subway [125], people's considerations of social acceptability may prioritize safety concerns. Interaction with situated visualizations in these scenarios should be considered a secondary task, which does not occupy too much attention [84].…”

Section: 12mentioning

confidence: 99%

Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public Environments

Zhu,

Wang,

Zeng

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Situated visualization blends data into the real world to fulfill individuals' contextual information needs. However, interacting with situated visualization in public environments faces challenges posed by users' acceptance and contextual constraints. To explore appropriate interaction design, we first conduct a formative study to identify users' needs for data and interaction. Informed by the findings, we summarize appropriate interaction modalities with eye-based, hand-based and spatially-aware object interaction for situated visualization in public environments. Then, through an iterative design process with six users, we explore and implement interactive techniques for activating and analyzing with situated visualization. To assess the effectiveness and acceptance of these interactions, we integrate them into an AR prototype and conduct a within-subjects study in public scenarios using conventional hand-only interactions as the baseline. The results show that participants preferred our prototype over the baseline, attributing their preference to the interactions being more acceptable, flexible, and practical in public. CCS CONCEPTS• Human-centered computing → Visualization techniques.

show abstract

Section: 12mentioning

confidence: 99%

Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public Environments

Zhu,

Wang,

Zeng

et al. 2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…Cultural beliefs and user stereotypes could also shape the social perceptions of wearables and user-worn recording devices [30,35,63]. Bystander considerations also play a role in social acceptability where interactions that provide an explanation [71] are likely to be better acceptable than fully hidden interactions. Future research efforts should focus on designing strategies to improve social acceptability, possibly following guidelines in social acceptability research in HCI [38].…”

Section: Privacy and Social Acceptabilitymentioning

confidence: 99%

Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation

Zulfikar,

Chan,

Maes

2024

Proceedings of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

People have to remember an ever-expanding volume of information. Wearables that use information capture and retrieval for memory augmentation can help but can be disruptive and cumbersome in real-world tasks, such as in social settings. To address this, we developed Memoro, a wearable audio-based memory assistant with a concise user interface. Memoro uses a large language model (LLM) to infer the user's memory needs in a conversational context, semantically search memories, and present minimal suggestions. The assistant has two interaction modes: Query Mode for voicing queries and Queryless Mode for on-demand predictive assistance, without explicit query. Our study of (N=20) participants engaged in a realtime conversation, demonstrated that using Memoro reduced device Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

show abstract

“…Whilst likely acceptable at home or in private, on a public bus such behaviour could be extremely unacceptable and even disruptive to other passengers. When interacting with technology that is highly visible to others, such as a headset occluding the face, the sustained spectatorship of other passengers creates a potentially uncomfortable situation for both users and passengers alike (Wiliamson et al 2011).…”

Section: Challenge 4: Acceptable Use In Shared Transitmentioning

confidence: 99%

Challenges in passenger use of mixed reality headsets in cars and other transportation

McGill

Williamson

et al. 2019

Virtual Reality

Self Cite

View full text Add to dashboard Cite

This paper examines key challenges in supporting passenger use of augmented and virtual reality headsets in transit. These headsets will allow passengers to break free from the restraints of physical displays placed in constrained environments such as cars, trains and planes. Moreover, they have the potential to allow passengers to make better use of their time by making travel more productive and enjoyable, supporting both privacy and immersion. However, there are significant barriers to headset usage by passengers in transit contexts. These barriers range from impediments that would entirely prevent safe usage and function (e.g. motion sickness) to those that might impair their adoption (e.g. social acceptability). We identify the key challenges that need to be overcome and discuss the necessary resolutions and research required to facilitate adoption and realize the potential advantages of using mixed reality headsets in transit.

show abstract

Multimodal mobile interactions

Cited by 39 publications

References 28 publications

Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public Environments

Make Interaction Situated: Designing User Acceptable Interaction for Situated Visualization in Public Environments

Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation

Challenges in passenger use of mixed reality headsets in cars and other transportation

Contact Info

Product

Resources

About