Hand segmentation and fingertip detection for interfacing of stereo vision-based smart glasses

Abstract

In this paper, we propose a hand segmentation and fingertip detection method for interacting with projected digital contents in the binocular smart glasses, with a built-in stereo cameras. In order to make robust hand segmentation under the various environmental conditions, we used the adaptive skin color model which is created by fusion of depth and color information. Also, we have defined the finger range to reduce time complexity and to find correct fingertip in the process of fingertip detection. Since it provides a stable hand gesture interface, it can be applied to various applications for smart glasses without the restriction of the mobility and the inconvenience.

Interaction Methods for Smart Glasses

“…Moreover, direct pointing or manipulation are analogue to the touch interface on a smartphone, that is, touchscreen, Interaction Methods for Smart Glasses 1:23 but the mouse interaction is not available on smart glasses and the visual content is no longer touchable. Therefore, pointing gestures become a viable option for 2D interfaces, for instance, Heo et al [49] proposes a vision-based pointing gesture system by detecting the number of fingertip, instead of identifying the silhouette of the hand posture. More importantly, virtual 3D objects are always involved in augmented reality.…”