GrabAR: Occlusion-aware Grabbing Virtual Objects in AR

Tang, Xiao; Hu, Xiaowei; Cohen–Or, Daniel

doi:10.1145/3379337.3415835

Cited by 24 publications

(10 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The addition of new real objects requires their modeling, but each time it is costly and labor-intensive. Furthermore, if a real object is deformed: shape and/or size (e.g., a moving animal, fallen doll, user's hand) at runtime, the system may either lost track or provide an incorrect interaction [22][23][24]. Incorrect interactions may reflect poorly on the user experience [5,25,26].…”

Section: Interacting With Real Objects In Armentioning

confidence: 99%

Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality

et al. 2021

View full text Add to dashboard Cite

Realistic interactions with real objects (e.g., animals, toys, robots) in an augmented reality (AR) environment enhances the user experience. The common AR apps on the market achieve realistic interactions by superimposing pre-modeled virtual proxies on the real objects in the AR environment. This way user perceives the interaction with virtual proxies as interaction with real objects. However, catering to environment change, shape deformation, and view update is not a trivial task. Our proposed method uses the dynamic silhouette of a real object to enable realistic interactions. Our approach is practical, lightweight, and requires no additional hardware besides the device camera. For a case study, we designed a mobile AR application to interact with real animal dolls. Our scenario included a virtual human performing four types of realistic interactions. Results demonstrated our method’s stability that does not require pre-modeled virtual proxies in case of shape deformation and view update. We also conducted a pilot study using our approach and reported significant improvements in user perception of spatial awareness and presence for realistic interactions with a virtual human.

show abstract

Section: Interacting With Real Objects In Armentioning

confidence: 99%

Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Their work supports occlusion representation calculation, registration error adaption, and adaptive content placement. Alternatively, Tang et al [227] present GrabAR, a system using a custom compact DNN for generating occlusion masks to support real-time grabbing of virtual objects in AR. The model is able to calculate and segment the hand to provide visually plausible interactions of objects in the virtual AR environment.…”

Section: For Adaptive Uismentioning

confidence: 99%

Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence

Cao,

Lam,

Lee

et al. 2021

Preprint

View full text Add to dashboard Cite

Mobile Augmented Reality (MAR) integrates computer-generated virtual objects with physical environments for mobile devices. MAR systems enable users to interact with MAR devices, such as smartphones and headworn wearables, and performs seamless transitions from the physical world to a mixed world with digital entities. These MAR systems support user experiences by using MAR devices to provide universal accessibility to digital contents. Over the past 20 years, a number of MAR systems have been developed, however, the studies and design of MAR frameworks have not yet been systematically reviewed from the perspective of user-centric design. This article presents the first effort of surveying existing MAR frameworks (count: 37) and further discusses the latest studies on MAR through a top-down approach: (1) MAR applications; (2) MAR visualisation techniques adaptive to user mobility and contexts; (3) systematic evaluation of MAR frameworks including supported platforms and corresponding features such as tracking, feature extraction plus sensing capabilities; and (4) underlying machine learning approaches supporting intelligent operations within MAR systems. Finally, we summarise the development of emerging research fields, current state-of-the-art, and discuss the important open challenges and possible theoretical and technical directions. This survey aims to benefit both researchers and MAR system developers alike. CCS Concepts: • Human-centered computing → Mixed / augmented reality; Graphical user interfaces; • Software and its engineering → Development frameworks and environments; • Computing methodologies → Machine learning approaches.

show abstract

“…Among others, the correct depiction of occlusion among objects (real or virtual) is important [16][17][18], whereas incorrect rendering is known to adversely affect the sense realism [19]. However, acquiring three-dimensional information of the environment and objects in it is not a trivial task [16,20], and as such, virtual objects are often added onto the real by simply overlaying them in the foreground of the screen space [21].…”

Section: Perceptual Issues In Armentioning

confidence: 99%

“…Recent advances in tracking [22,23] and depth estimation techniques [21,24,25] are becoming more amenable even on mobile platforms. However, gracefully handling unknown objects unexpectedly introduced into the scene, such as CIROs, is still a difficult task, especially on the relatively computationally limited handheld AR devices.…”

Section: Perceptual Issues In Armentioning

confidence: 99%

CIRO: The Effects of Visually Diminished Real Objects on Human Perception in Handheld Augmented Reality

Kim

Lee

et al. 2021

Electronics

View full text Add to dashboard Cite

Augmented reality (AR) scenes often inadvertently contain real world objects that are not relevant to the main AR content, such as arbitrary passersby on the street. We refer to these real-world objects as content-irrelevant real objects (CIROs). CIROs may distract users from focusing on the AR content and bring about perceptual issues (e.g., depth distortion or physicality conflict). In a prior work, we carried out a comparative experiment investigating the effects on user perception of the AR content by the degree of the visual diminishment of such a CIRO. Our findings revealed that the diminished representation had positive impacts on human perception, such as reducing the distraction and increasing the presence of the AR objects in the real environment. However, in that work, the ground truth test was staged with perfect and artifact-free diminishment. In this work, we applied an actual real-time object diminishment algorithm on the handheld AR platform, which cannot be completely artifact-free in practice, and evaluated its performance both objectively and subjectively. We found that the imperfect diminishment and visual artifacts can negatively affect the subjective user experience.

show abstract

GrabAR: Occlusion-aware Grabbing Virtual Objects in AR

Cited by 24 publications

References 45 publications

Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality

Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality

Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence

CIRO: The Effects of Visually Diminished Real Objects on Human Perception in Handheld Augmented Reality

Contact Info

Product

Resources

About