Hand and foot gesture interaction for handheld devices

Abstract

In this paper we present hand and foot based immersive multimodal interaction approach for handheld devices. A smart phone based immersive football game is designed as a proof of concept. Our proposed method combines input modalities (i.e. hand & foot) and provides a coordinated output to both modalities along with audio and video. In this work, human foot gesture is detected and tracked using template matching method and Tracking-Learning-Detection (TLD) framework. We evaluated our system's usability through a user study in which we asked participants to evaluate proposed interaction method. Our preliminary evaluation demonstrates the efficiency and ease of use of proposed multimodal interaction approach.

Hand-free motion interaction on Google Glass

“…We have performed extensive test to confirm the intuitive and robustness of the hand/foot detection and tracking algorithm [Lu et al 2013] ] as well as evaluate the usability of the designed gestures and social acceptability of the demonstrations on a smartphone based hybrid wearable framework [Lv 2013], the designed touch-less gesture were perceived as 'applicable' and 'acceptable'. Future work will consist in tracking multiple body parts, for instance by applying multiple fingers tracking algorithms in parallel while keeping performance high.…”

“…Presented application allows user to perform touch-less interaction by hand or foot gesture in front of the camera of google glass. Based on the same core ubiquitous gestures recognition algorithm as used in this demonstration, a hybrid wearable smartphone system based on mixed hardware and software has been presented in our previous work [Lv 2013] [Lu et al 2013] , which can support either hand or foot interaction with today' smartphone.…”

Multimodal Hand and Foot Gesture Interaction for Handheld Devices

“…The football application was presented by and Lv et al [2014]. We also extended this method to other parts of the human body preliminarily, such as to the hand [Lu and ur Réhman 2013;]. …”

Use of Ecological Gestures in Soccer Games Running on Mobile Devices

“…In the method proposed in [15] the main problem was the Kinect sensor which is by definition a non-mobile sensor in terms of computation weight and energy consumption. Paelke et al [18] and later Lu et al [19] have used computer vision like in [15] but change the sensor for the mobile's back cameras in order to recognize the foot and track its movements. There is still a series of problems with this method directly related to the use of vision.…”