Visual rendering of shapes on 2D display devices guided by hand gestures

Abstract

Designing of touchless user interface is gaining popularity in various contexts. Using such interfaces, users can interact with electronic devices even when the hands are dirty or nonconductive. Also, user with partial physical disability can interact with electronic devices using such systems. Research in this direction has got major boost because of the emergence of low-cost sensors such as Leap Motion, Kinect or RealSense devices. In this paper, we propose a Leap Motion controller-based methodology to facilitate rendering of 2D and 3D shapes on display devices. The proposed method tracks finger movements while users perform natural gestures within the field of view of the sensor. In the next phase, trajectories are analyzed to extract extended Npen++ features in 3D. These features represent finger movements during the gestures and they are fed to unidirectional left-to-right Hidden Markov Model (HMM) for training. A one-to-one mapping between gestures and shapes is proposed. Finally, shapes corresponding to these gestures are rendered over the display using MuPad interface. We have created a dataset of 5400 samples recorded by 10 volunteers. Our dataset contains 18 geometric and 18 non-geometric shapes such as "circle", "rectangle", "flower", "cone", "sphere" etc. The proposed methodology achieves an accuracy of 92.87% when evaluated using 5-fold cross validation method. Our experiments revel that the extended 3D features perform better than existing 3D features in the context of shape representation and classification. The method can be used for developing useful HCI applications for smart display devices.

Accuracy of Hidden Markov Models in Identifying Alterations in Movement Patterns during Biceps-Curl Weight-Lifting Exercise

“…This study examined whether an HMM trained using different types of coordinates derived from the same movement is able to detect significant variations in shoulder, elbow and load positional sequence, and therefore changes in movement pattern of the bicepscurl exercise. As demonstrated in Table 2, linear topology HMM, trained with Cartesian coordinates [47,61] and corrected using environmental measurements, was able to detect significant adjustments in movement patterns in the sagittal plane, and this was confirmed by ANOVA. Therefore, the findings evidenced the confidence in HMM for detecting undesired joint positional adjustments when comparing the standard reference of simple human lift movement (e.g., single-joint action with no load) to the lift attempts with heavy loads.…”

Deep learning based spatio‐temporal hand gesture recognition system in complex environment

“…The hand tracker used contour points and Harris corners, a pixel‐based skin detection method to recover the tracked hand in subsequent frames based on information from the hand detector and wrist position estimator. The normalised sequence of captured 3D space coordinates was taken as input, and the sequence of features was computed along the trajectory (Singla et al, 2019). Gesture direction, curvature, aspect, curliness, slope, and lineness were some of the features that had been calculated and used to develop feature space and recognition.…”

HyperCube4x: A viewport management system proposal

“…38 Despite its popularity, touchscreens are not a solution when the device is not close enough (such as smart TV), or the hands are dirty or not conductive. 40 As such, both methods may have to be available to the user.…”