Hand segmentation for hand-object interaction from depth map

Abstract

Hand segmentation for hand-object interaction is a necessary preprocessing step in many applications such as augmented reality, medical application, and human-robot interaction. However, typical methods are based on color information which is not robust to objects with skin color, skin pigment difference, and light condition variations. Thus, we propose hand segmentation method for hand-object interaction using only a depth map. It is challenging because of the small depth difference between a hand and objects during an interaction. To overcome this challenge, we propose the two-stage random decision forest (RDF) method consisting of detecting hands and segmenting hands. To validate the proposed method, we demonstrate results on the publicly available dataset of hand segmentation for hand-object interaction. The proposed method achieves high accuracy in short processing time comparing to the other state-of-the-art methods.

DenseAttentionSeg: Segment hands from interacted objects using depth input

“…Kang et al adopted a simple segmentation method by using a black wristband in their hand tracking work [15]. They also studied the problem of hand segmentation recently [2].…”

“…Hand and object interaction: Though the most common situation of hands in our daily life is to interact with all kinds of objects, it has not been fully researched. Kang et al proposed a two-stage RDF to segment hand and object on depth [2]. Their solution runs fast while with a fair accuracy.…”

Random Forest With Learned Representations for Semantic Segmentation

“…Since the probability p c (c|x) is predicted for each pixel independently, the probability can be stabilized considering nearby predictions [7], [55]. In this paper, we apply simple modified bilateral filter [7], [56] that processes weighted averaging of the probabilities of the data points in close distance and similar intensity on the input X. The filtering is defined as follows:…”

“…To achieve accurate and efficient pixel-wise classification, Shotton et al presented a random forest-based method and applied it for semantic segmentation and body pose estimation in [1]- [3]. This work has been broadly employed in many related applications [4]- [7] and in Microsoft Kinect [8]. To improve accuracy in semantic segmentation, convolutional neural network-based methods have been proposed in [9]- [14].…”

“…Sharp et al employed the memory-efficient random forest method [43] to predict initial hand pose in hand tracking [5]. Kang et al proposed the two-stage random forest method consisting of detection and segmentation for hand segmentation in handobject interaction [7].…”