Depth Camera-Based 3D Hand Gesture Controls with Immersive Tactile Feedback for Natural Mid-Air Gesture Interactions

Kim, K.; Kim, Joongrock; Choi, Jae-Sung; Kim, Jung-Hyun; Lee, Sangyoun

doi:10.3390/s150101022

Cited by 46 publications

(23 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We thus study the augmentation or combination of tactile input with other interaction paradigms to address its limits. In the past, tactile input has been combined with spatially aware tangible devices [SBD12, SvZP∗16, BIAI17a], mid‐air gestures [KKC∗15], and pressure input (e.g., [BH09, HL12, CDVB17]) to offer more interaction possibilities. While these techniques have been positively evaluated, most of them rely on custom‐made sensing units which limit their adoption.…”

Section: Introductionmentioning

confidence: 99%

Augmenting Tactile 3D Data Navigation With Pressure Sensing

Wang

Besançon²,

Ammi

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

We present a pressure‐augmented tactile 3D data navigation technique, specifically designed for small devices, motivated by the need to support the interactive visualization beyond traditional workstations. While touch input has been studied extensively on large screens, current techniques do not scale to small and portable devices. We use phone‐based pressure sensing with a binary mapping to separate interaction degrees of freedom (DOF) and thus allow users to easily select different manipulation schemes (e. g., users first perform only rotation and then with a simple pressure input to switch to translation). We compare our technique to traditional 3D‐RST (rotation, scaling, translation) using a docking task in a controlled experiment. The results show that our technique increases the accuracy of interaction, with limited impact on speed. We discuss the implications for 3D interaction design and verify that our results extend to older devices with pseudo pressure and are valid in realistic phone usage scenarios.

show abstract

Section: Introductionmentioning

confidence: 99%

Augmenting Tactile 3D Data Navigation With Pressure Sensing

Wang

Besançon²,

Ammi

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Previous research has shown that haptic feedback has the potential to improve mid-air gestural interaction [8]. In this paper, we focus our research on the localisation of virtual widgets that are reachable by hands.…”

Section: B! Haptic Feedback For Mid-air Gesturesmentioning

confidence: 99%

“…Using piezoelectric actuators, Kim et al provided participants with haptic feedbacks to inform them of the start, the end and the recognition states of a gesture [8]. Comparing with a situation where no feedback was provided, they showed that haptic feedback increased the accuracy and reduced the trajectory length of the gesture, and potentially users' fatigue.…”

Section: B! Haptic Feedback For Mid-air Gesturesmentioning

confidence: 99%

See 1 more Smart Citation

Touching the invisible: Localizing ultrasonic haptic cues

Brewster

2015

2015 IEEE World Haptics Conference (WHC)

View full text Add to dashboard Cite

Abstract-While mid-air gestures offer new possibilities to interact with or around devices, some situations, such as interacting with applications, playing games or navigating, may require visual attention to be focused on a main task. Ultrasonic haptic feedback can provide 3D spatial haptic cues that do not demand visual attention for these contexts. In this paper, we present an initial study of active exploration of ultrasonic haptic virtual points that investigates the spatial localization with and without the use of the visual modality. Our results show that, when providing haptic feedback giving the location of a widget, users perform 50% more accurately compared to providing visual feedback alone. When provided with a haptic location of a widget alone, users are more than 30% more accurate than when given a visual location. When aware of the location of the haptic feedback, active exploration decreased the minimum recommended widget size from 2cm 2 to 1cm 2 when compared to passive exploration from previous studies. Our results will allow designers to create better mid-air interactions using this new form of haptic feedback. I.! INTRODUCTIONSophisticated and affordable sensors have lead many to consider touchless gestural interaction in new application contexts such as desktop computers, gaming, interactive tabletops or inside cars [12,14,17]. In addition to simple tasks such as pan and zoom or switching programs, gestures above the keyboard allow users to manipulate complex widgets without the burden of using another device, such as a mouse. For example, it is possible to enable a colour picker widget with a key on the keyboard and control the wheel with mid-air gestures [17]. Mid-air gestures also provide rich interaction techniques to execute sophisticated commands and give six degrees of freedom with which to manipulate digital content [12]. However, a key limitation with the research and products in this area is that they provide no haptic feedback; users can gesture but they cannot feel the controls they are interacting with.Gestures above the keyboard can limit the increasing complexity of keyboards by addressing special functionalities such as media controls keys, numeric keypads, specific or additional keyboard layouts, shortcuts, or even specific input controllers like sliders or dials. For example, introducing layers of configurable virtual input controllers in the space around the keyboard would enable a rich set of complementary controllers for secondary or specific tasks (Figure 1). However, this requires the ability to locate such controllers * dong-bach.vo@glasgow.ac.uk † stephen.brewster@glasgow.ac.uk easily and without decreasing user performance by cluttering the display with visual information about where they are.

show abstract

“…These technologies include cameras and computer vision and room-based or built-in sensors such as cameras [6], PointGrab [7], laptops [8], smartphones [9,10], GPS [11], vision-based [12], and accelerometers [13,14], as well as sensors worn on the human body-such as rings [15], armbands [16], and wristbands [17,18]. However, they have drawbacks.…”

Section: Device-based Gesture Recognition Systemsmentioning

confidence: 99%

WiGeR: WiFi-Based Gesture Recognition System

Al-qaness

2016

IJGI

View full text Add to dashboard Cite

Abstract:Recently, researchers around the world have been striving to develop and modernize human-computer interaction systems by exploiting advances in modern communication systems. The priority in this field involves exploiting radio signals so human-computer interaction will require neither special devices nor vision-based technology. In this context, hand gesture recognition is one of the most important issues in human-computer interfaces. In this paper, we present a novel device-free WiFi-based gesture recognition system (WiGeR) by leveraging the fluctuations in the channel state information (CSI) of WiFi signals caused by hand motions. We extract CSI from any common WiFi router and then filter out the noise to obtain the CSI fluctuation trends generated by hand motions. We design a novel and agile segmentation and windowing algorithm based on wavelet analysis and short-time energy to reveal the specific pattern associated with each hand gesture and detect duration of the hand motion. Furthermore, we design a fast dynamic time warping algorithm to classify our system's proposed hand gestures. We implement and test our system through experiments involving various scenarios. The results show that WiGeR can classify gestures with high accuracy, even in scenarios where the signal passes through multiple walls.

show abstract

Depth Camera-Based 3D Hand Gesture Controls with Immersive Tactile Feedback for Natural Mid-Air Gesture Interactions

Cited by 46 publications

References 22 publications

Augmenting Tactile 3D Data Navigation With Pressure Sensing

Augmenting Tactile 3D Data Navigation With Pressure Sensing

Touching the invisible: Localizing ultrasonic haptic cues

WiGeR: WiFi-Based Gesture Recognition System

Contact Info

Product

Resources

About