A Fast Automatic Holoscopic 3D Micro-gesture Recognition System for Immersive Applications

Abstract

Immersive technology attempts to emulate a physical world through the means of a digital or simulated world. Micro-gestures are small variation actions on human hands defined by user that is one of the most convenient human action in immersive technology. Holoscopic 3D imaging uses bionics technology to capture spatial image in the pattern of fly's eye and it has fruitful 3D cubic information compared to 2D images that can be used for high accurate micro-gesture controller systems.In this paper, a new micro-gesture recognition system based on holoscopic 3D imaging system is proposed for immersive applications. It is built on fast pre-processing, dynamic image feature extraction and a nonlinear Support Vector Machine classifier. It is evaluated on the public Holoscopic Micro 3D Gesture (HoMG) dataset outperforming all the existing state-of-the-art methods on the same dataset.

A Novel pseudo viewpoint based Holoscopic 3D Micro-gesture Recognition

“…Zhang et al [12] introduced image-based subset features together with the video-based data and combined several convolutional neural networks to obtain the final recognition result. Recently, Qin et al [5] proposed a method by combining LPQTOP features and a non-liner SVM classifier for recognition.…”

“…82.0 Zhang et al [12] SeResNet50+M.V. 82.0 Qin et al [5] LPQTOP+n.SVM+mRMR 84.6 This work PVP25+D3D+M.V. 85.2…”

“…Finally, they achieved an accuracy of 82% accuracy. The existing sate of the the art performance is reported by [5], which combined dynamic image feature extraction and a non-linear Support Vector Machine (SVM) classifier to evaluate HoMG database. Notably our final result is 85.4%, which surpass all prior results shown in the table.…”

“…Hand gestures have been widely used for natural human-computer interaction (HCI) over past decades. As an effective manner to capture the various types of both normal and micro gesture, holoscopic 3D (H3D) camera has attracted increasingly attention in recent years [12][6] [8][3][10] [5]. One particular advantage of H3D cameras, compared with most existing sensors such as Kinect and Leap Motion, is that 3D objects could be captured in 2D format by the special microlens-array of the camera sensor.…”