Augmented Reality Humans: Towards Multisensorial Awareness

Nijholt, Anton

doi:10.1007/978-3-030-92909-1_16

Cited by 5 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Techniques from computer vision, machine learning, and natural language processing are employed to develop effective ways for humans to interact with autonomous systems and for the systems to understand better and respond to human behavior [61], [62]. This ultimately means that communication happens over many modalities including textual [63], [64], visual [65], [66], auditory [48], haptic [67], [68], augmented reality, and virtual reality [69]- [71], and multimodal mediums [17], [72] depending on the task and HAT needs.…”

Section: A Human-autonomy Interaction and Communicationmentioning

confidence: 99%

Human-Agent Teaming: A System-Theoretic Overview

Meimandi,

Bolton,

Beling

2024

Preprint

View full text Add to dashboard Cite

show abstract

Section: A Human-autonomy Interaction and Communicationmentioning

confidence: 99%

Human-Agent Teaming: A System-Theoretic Overview

Meimandi,

Bolton,

Beling

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, the virtual dog in our AR example is a quasi-other. More generally, we can have virtual humans as quasi-others in the virtual layer (Nijholt, 2021b) with whom we can interact.…”

Section: Human -(Smart-and-ar) Technology Relationsmentioning

confidence: 99%

Augmented Reality: Beyond Interaction

Nijholt¹

2022

Human Factors in Virtual Environments and Game Design

View full text Add to dashboard Cite

In 1997 Ronald T. Azuma introduced the following definition of Augmented Reality (AR): “Some researchers define AR in a way that requires the use of Head-Mounted Displays (HMDs). To avoid limiting AR to specific technologies, this survey defines AR as systems that have the following three characteristics: 1) Combines real and virtual, 2) Interactive in real-time, 3) Registered in 3-D.” Azuma also mentions that “AR might apply to all senses, not just sight.” [1] This definition has been leading in AR research until now. AR researchers focused on the various ways technology, in particular digital technology (computer-generated imagery, computer vision and world modelling, interaction technology, and AR display technology), could be developed to realize this AR view. The emphasis has been on addressing the sight sense when generating and aligning virtual content, our most dominant sense, although we can not survive without the others. Azuma and others mention the other senses and assume that this definition also covers other than computer-generated imagery, per-haps even other than computer generated and (spatial-temporal) generated and con-trolled virtual content. Nevertheless, the definition has some constituents that can be given various interpretations. This makes it workable, but it is useful to discuss how we should distinguish between real and virtual content, what is it that distinguishes real from virtual, or how virtual content can trigger changes in the real world (and the other way around), take into account that AR becomes part of ubiquitous computing. That is, rather than looking at AR from the point of view of particular professional, educational, or entertaining applications, we should look at AR from the point of view that it is ever-present, and embedded in ubiquitous computing (Ubicomp), and having its AR devices’ sensors and actuators communicate with the smart environments in which it is embedded.The focus in this paper is on ‘optical see-through’ (OSR) AR and ever-present AR. Ever-present AR will become possible with non-obtrusive AR glasses [2] or contact lenses [3,4]. Usually, interaction is looked upon from the point of view of what we see and hear. But we certainly are aware of touch experiences and exploring objects with active touch. We can also experience scents and flavors, passively but also actively, that is, consciously explore scents or tastes, become aware of them, and ask the environment, not necessarily explicitly since our preferences are known and our intentions can be predicted, to respond in an appropriate way to evoke or continue an interaction.Interaction in AR and with AR technology requires a new look at interaction. Are we interacting with the AR device, with the environment, or with the environment through the AR device? Part of what we perceive is real, part of what we perceive is superimposed on reality, and part of what we perceive is the interaction between real and virtual reality. How to interact with this mix of realities? Additionally, our HMD AR provides us with view changes because of position and head orientation or gaze changes. We interact with the device with, for example, speech and hand gestures, we interact with the environment with, for example, pose changes, and we interact with the virtual content with interaction modalities that are appropriate for that content: push a virtual block, open a virtual door, or have a conversation with a virtual hu-man that inhabits the AR world. In addition, we can think of interactions that be-come possible because technology allows us to get access and act upon sensor information that cannot be perceived with our natural perception receptors. In a ubiquitous computing environment, our AR device can provide us with a 360 degrees view of our environment, drones can feed us with information from above, infrared sensors know about people and events in the dark, our car receives visual information about not yet visible vehicles approaching an intersection [5], sound frequencies be-yond the human ear can be made accessible, smell sensors can enhance the human smell sense, et cetera.In this paper, we investigate the characteristics of interactions in AR and relate them to the regular human-computer interaction characteristics (interacting with tools) [6], interaction with multimedia [7] interaction through behavior [8], implicit interaction [9], embodied interaction [10], fake interaction [11], and interaction based on Gibson’s visual perception theory [12]. This will be done from the point of view of ever-present AR [13] with optical see-through wearable devices.References could not be included because of space limitations.

show abstract

“…In (OST) AR these virtual humans will accompany us or will pop up with task-and context-related knowledge and we can have or develop different types of relationships with them, for example, a domestic servant/employer (butler), buyer/seller, friendship, or love relationship. Presently we see these virtual humans also appear in the AR context [6]. Unlike avatars that are intended to directly depict the behavior of its human owner (obtained from sensors such as cameras, motion captures systems, microphones, et cetera), an 'autonomous' embodied agent such as a virtual human has its perceptive capabilities, behavior, and intelligence arise from algorithms.…”

Section: Background: Towards Social Armentioning

confidence: 99%

Social Augmented Reality: A Multiperspective Survey

Nijholt

2021

2021 Joint 10th International Conference on Informatics, Electronics &Amp; Vision (ICIEV) and 2021 5th International Conference

Self Cite

View full text Add to dashboard Cite

We introduce research on social augmented reality with an emphasis on social face-to-face interaction. In social interaction interactants employ knowledge about their conversational partner and have their verbal interaction supported by nonverbal interaction cues. We survey the issues that arise when we pursue social interaction in augmented reality. Since handhelds and bulky head-mounted devices hardly allow unobtrusive interaction we pay extensive attention to developments in the field of smart glasses and smart contact lenses. The focus is on the use of smart augmented reality glasses during social interactions. Acceptance issues and disruption of social interaction due to the use of these devices are also touched upon. Contribution-An overview of social aspects of interpersonal communication in augmented reality applications.

show abstract

Augmented Reality Humans: Towards Multisensorial Awareness

Cited by 5 publications

References 36 publications

Human-Agent Teaming: A System-Theoretic Overview

Human-Agent Teaming: A System-Theoretic Overview

Augmented Reality: Beyond Interaction

Social Augmented Reality: A Multiperspective Survey

Contact Info

Product

Resources

About