Privacy-Enhancing Technology and Everyday Augmented Reality

Abstract

Fundamental to Augmented Reality (AR) headsets is their capacity to visually and aurally sense the world around them, necessary to drive the positional tracking that makes rendering 3D spatial content possible. This requisite sensing also opens the door for more advanced AR-driven activities, such as augmented perception, volumetric capture and biometric identification - activities with the potential to expose bystanders to significant privacy risks. Existing Privacy-Enhancing Technologies (PETs) often safeguard against these risks at a low level e.g., instituting camera access controls. However, we argue that such PETs are incompatible with the need for always-on sensing given AR headsets' intended everyday use. Through an online survey (N=102), we examine bystanders' awareness of, and concerns regarding, potentially privacy infringing AR activities; the extent to which bystanders' consent should be sought; and the level of granularity of information necessary to provide awareness of AR activities to bystanders. Our findings suggest that PETs should take into account the AR activity type, and relationship to bystanders, selectively facilitating awareness and consent. In this way, we can ensure bystanders feel their privacy is respected by everyday AR headsets, and avoid unnecessary rejection of these powerful devices by society.

A Viewpoint on the Societal Impact of Everyday Augmented Reality and the Need for Perceptual Human Rights

“…Everyday AR would become a large-language-model-driven personalized expert on our shoulder, seemingly augmenting our intelligence and cognition. 1,2 Everyday AR will empower users, communities, business, governments, and other entities to alter, augment, diminish, or otherwise mediate our perception of people, places, objects, media, and more.…”

“…T he personal computing landscape is on the verge of a transition: from the 2D surfaces of smartphones, monitors, and other "physical" displays, to the ethereal spatial computing of augmented, mixed, and extended reality (AR/MR/XR). 1 Currently, AR headsets, glasses, and more (hereafter referred to collectively as AR headsets) come equipped with a variety of sensing that drives their capability to understand the world around them, for example, packing eye tracking, outward-facing red, green, blue plus depth sensing, directional microphone arrays, etc. 2 into wearable form factors.…”

“…1 Currently, AR headsets, glasses, and more (hereafter referred to collectively as AR headsets) come equipped with a variety of sensing that drives their capability to understand the world around them, for example, packing eye tracking, outward-facing red, green, blue plus depth sensing, directional microphone arrays, etc. 2 into wearable form factors. Coupled with their ability to render virtual visual and auditory augmentations around a user, these headsets present the foundations for a transformative consumer technology-supporting personal augmentation of intelligence, perception, cognition, and more.…”

“…Everyday AR headsets will place themselves between our eyes/ears and our surrounding reality, mediating our perception of reality throughout our daily lives. 1 In the process, everyday AR will empower users, communities, business, governments, and other entities to alter, augment, diminish, or otherwise mediate our perception of people, places, objects, media, and more.…”

The Social Impact of Extended Reality Spatial Productivity in Constrained, Public and Passenger Spaces

“…Extended Reality (XR), and in-particular the advent of everyday, ubiquitous Augmented Reality (AR) [5,11,17] promises the capability to be productive anywhere, anytime -freeing users from the restrictions of physical displays and even input peripherals by transitioning from physical material computing towards spatial computing -where displays and apps are virtual and inputs might be derived from body-based interactions, what we term spatial productivity. Our early research into wide virtual workspaces [7] exemplified the potential here by examining the ergonomic impact that virtual workspaces would have on users head/neck movements, and how to minimize that impact through e.g.…”