Bare-Hand Depth Perception Used in Augmented Reality Assembly Supporting

Abstract

Augmented Reality (AR) had been used in mechanical assembly supporting. The current reported researches of AR assembly supporting system (ARASS) less concerned about the interaction between the operators' hands and virtual objects, especially on the occlusion between hands and virtual objects. The correct occlusion results would make ARASS more immersive and then the operators' interaction experiences were increased. To address this issue, this paper presented a bared-hand depth perception method that was designed for improving interaction experience in ARASS. The method was based on an interaction scene that was designed for AR mechanical assembly supporting. The method consisted of two components: hand segmentation and hand mesh generation. Hand segmentation method segmented operator's hand areas from depth image of scene and divided hand area into several sub areas that provided hand information. Hand surface mesh generation method generated hand surface meshes in 3D space based on the results of hand segmentation. The meshes were used to solve the occlusion problem of hand area in AR scene. The results verified that the bared-hand depth perception method could handle the occlusion between operator's hands and virtual objects correctly in real-time and recognize hand information in a limited space. The method could increase operators' depth perception of bared hand and make ARASS more immersive. INDEX TERMS Augmented assembly, augmented reality, hand depth perception.

Design and implementation of virtual fitting system based on gesture recognition and clothing transfer algorithm

“…Xu et al 19 proposed a gesture recognition method based on semi-supervised joint learning, using color segmentation algorithm to separate the contour of hand from the image and realize gesture recognition. Liang et al 20 presented a bared-hand depth perception method that was designed for improving interaction experience in ARASS and applied it to the augmented reality assembly system. Chen et al 21 proposed a robust real-time dynamic gesture recognition system for natural interaction with service robots in a dynamic environment.…”

A Novel Suture Training System for Open Surgery Replicating Procedures Performed by Experts Using Augmented Reality

“…2.79b (The Blender Foundation, Amsterdam, Netherlands) that changes the angle between the forceps' tips in the 3D-CG model. When overlaying information in AR applications, occlusion, the ability to hide 3D-CG objects located behind other objects, plays an essential role in providing the right spatial positional relationship [18]. A hand occlusion function was provided to maintain the relationship between hands and 3D-CG models in our application.…”

Design and analysis of depth cues on depth perception in interactive mixed reality simulation systems

“…Among them, occlusion is the most potent 17 . X‐ray vision 18,19 and the grid 20 are effective ways to deal with depth‐sorting distortion caused by occlusion. Compared to occlusion, shadow casting is a more detailed depth cue.…”