FPSI-Fingertip pose and state-based natural interaction techniques in virtual environments

Rehman, Inam Ur; Ullah, Sadiq; Khan, Dawar

doi:10.1007/s11042-022-13824-w

Cited by 3 publications

(1 citation statement)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some works demonstrate innovative approaches to interaction within virtual environments, significantly reducing user effort. Rehman, Ullah & Khan (2023) propose four techniques that rely on the pose of a single fingertip. It makes interaction efficient requiring minimal learning effort for users.…”

Section: Related Workmentioning

confidence: 99%

The real-time hand and object recognition for virtual interaction

Nuralin,

Daineko,

Aljawarneh

et al. 2024

PeerJ Computer Science

View full text Add to dashboard Cite

Recognizing hand-object interactions presents a significant challenge in computer vision. It arises due to the varying nature of hand-object interactions. Moreover, estimating the 3D position of a hand from a single frame can be problematic, especially when the hand obstructs the view of the object from the observer’s perspective. In this article, we present a novel approach to recognizing objects and facilitating virtual interactions, using a steering wheel as an illustrative example. We propose a real-time solution for identifying hand-object interactions in eXtended reality (XR) environments. Our approach relies on data captured by a single RGB camera during a manipulation scenario involving a steering wheel. Our model pipeline consists of three key components: (a) a hand landmark detector based on the MediaPipe cross-platform hand tracking solution; (b) a three-spoke steering wheel model tracker implemented using the faster region-based convolutional neural network (Faster R-CNN) architecture; and (c) a gesture recognition module designed to analyze interactions between the hand and the steering wheel. This approach not only offers a realistic experience of interacting with steering-based mechanisms but also contributes to reducing emissions in the real-world environment. Our experimental results demonstrate the natural interaction between physical objects in virtual environments, showcasing precision and stability in our system.

show abstract

Section: Related Workmentioning

confidence: 99%