VICs: A Modular Vision-Based HCI Framework

Abstract

Abstract. Many Vision-Based Human-Computer Interaction (VB-HCI) systems are based on the tracking of user actions. Examples include gazetracking, head-tracking, finger-tracking, and so forth. In this paper, we present a framework that employs no user-tracking; instead, all interface components continuously observe and react to changes within a local image neighborhood. More specifically, components expect a pre-defined sequence of visual events called Visual Interface Cues (VICs). VICs include color, texture, motion and geometric elements, arranged to maximize the veridicality of the resulting interface element. A component is executed when this stream of cues has been satisfied. We present a general architecture for an interface system operating under the VIC-Based HCI paradigm, and then focus specifically on an appearance-based system in which a Hidden Markov Model (HMM) is employed to learn the gesture dynamics. Our implementation of the system successfully recognizes a button-push with a 96% success rate. The system operates at frame-rate on standard PCs.

Face detection for automatic exposure control in handheld camera

“…Many workable software and hardware systems have been proposed for surveillance, robotic, human-computer interaction (HCI) [14], and intelligent traffic measurement [4]. Besides conventional vision applications, popularity of handheld devices with cameras creates potential for fantastic new applications in PDA's, cell phones, or any small battery driven device.…”

VisHap: augmented reality combining haptics and vision

“…Instead we perform efficient and robust appearance-based skin segmentation on the color image [11]. The basic idea of the appearance model is to split the image into small image tiles and build a hue histogram for each of the image patches.…”

“…For future images, we also build the histogram for image patches in the region of interest and carry out pairwise histogram comparison with the background model. Histogram intersection [11] is an efficient way to match histograms.…”

Visual Modeling of Dynamic Gestures Using 3D Appearance and Motion Features

“…Gestures have been one of the important interaction media in current humancomputer interaction (HCI) systems [3,4,11,12,14,16,18,22,25,26,28]. Furthermore, for 3-D virtual environments (VE) in which the user manipulates 3-D objects, gestures are more appropriate and potentially powerful than traditional interaction media, such as a mouse or a joystick.…”

“…Instead, we follow the site-centered computation fashion of Visual Interaction Cues (VICs) paradigm [3,25].…”