Gesture Recognition Using 3D Appearance and Motion Features

Abstract

We present a novel 3D

A robust method for VR-based hand gesture recognition using density-based CNN

“…Until recently, hand gesture recognition has been developed for various purposes, such as sign language understanding [1][2][3][4][5], Human-Computer Interaction [6], virtual environment interaction [4,[7][8][9] and controlling using robot [10][11][12]. Some applications using hand gestures as navigators to walk through a virtual environment [7,8], virtual keyboard, controller appliances or device inside a certain space [10][11][12], controller robot surgery and used in medical purposes application such as physical treatment [6].…”

“…There are two kinds of motion sensors, wearable sensors and camera-based sensors. In case using a camera-based sensor, from frames captured over time, the displacement of human joints position will be considered as human motion [9][10][11][20][21][22][23][24]. Motion with certain patterns will be understood as a gesture.…”

“…Generally, hand gestures will be categorized as hand pose, hand sequence of movement or hand trajectory, and hand continuous movement [2,3,6,9]. A hand pose is considerably simple, easy to be captured and recognized but not many poses can be represented using one hand or double hand without ambiguity [1, 6-8, 10, 12].…”

“…Trajectory gesture consists of several different poses to represent a whole gesture while in continuous gesture, poses and displacement positions or just one joint movement are considered one single gesture. However, trajectory and continuous movement consist of several poses, direction and orientation changing [3,5,6,9,11,20]. Since no duration limitation in performing a gesture, it needs duration normalization.…”

“…Since no duration limitation in performing a gesture, it needs duration normalization. Dynamic Time Wrapping (DTW) [25] or define a fix data sampling [9] can be used as solutions for the duration problem.…”