Thumb Inclination-Based Manipulation and Exploration, a Machine Learning Based Interaction Technique for Virtual Environments

Abstract

In the context of Virtual Reality (VR), interactions refer to the plausible actions in a Virtual Environment (VE). To have an engrossing interface, interactions by the gestures of hand are becoming prominent. With thisresearch work, a novel interaction technique is proposed where interactions are performed on the basis of the position of thumb in dynamic image stream. The technique needs no expensive tracker but an ordinary camera to trace hand movements and position of thumb. The interaction tasks are enacted in distinct interaction states, where the Angle of Inclination (AOI) of thumb is used for state-to-state transition. The angle is computed dynamically between the tip-of-thumb and the base of the Region of Interest (ROI) of an input image. The technique works in two phases: learning phase and application phase. In the learning phase, user-defined fist-postures with distinct AOI are learnt. The Support Vector Machine (SVM) classifier is trained by the AOI of the postures. In the application phase, interactions are performed in distinct interaction states whereas a particular state is activated by posing the known posture. To follow the trajectory of thumb, dynamic mapping is performed to control a virtual hand by the position of thumb in the input image. The technique is implemented in a Visual Studio project called Thumb-Based Interaction for Virtual Environments (TIVE). The project was evaluated by a group of 15 users in a moderate lighting condition. The 89.7% average accuracy rate of the evaluation proves suitability of the technique in the wide range VR applications.

The real-time hand and object recognition for virtual interaction

“…Furthermore, as the use of extra devices decreases, additional gestures are gradually being removed from virtual environment interaction ( Raees et al., 2021 ). The authors propose the thumb inclination method of manipulation within created media.…”