A Multimodal framework for Interacting with Virtual Environments

Sharma, Rajeev; Huang, Thomas S.; Pavlović, Vladimir

doi:10.1007/978-1-4613-1447-9_5

Cited by 16 publications

(10 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One approach is the fusion of color, motion and other visual cues [7] or the fusion of visual cues with nonvisual cues like speech or gaze [83]. The potential advantage of the socalled multimodal approach has not yet been fully exploited for hand localization though it has been explored for face localization in video [38].…”

Section: Localizationmentioning

confidence: 99%

“…For example, instead of designing a complicated gestural command for the object selection which may consist of a deictic gesture followed by a symbolic gesture (to symbolize that the object that was pointed at by the hand is supposed to be selected) a simple concurrent deictic gesture and verbal command "this" can be used [84]. The number of studies that explore the use of multimodality in HCI has been steadily increasing over the past couple of years [36], [83], [98], [99], [102]. At the present time, the integration of communication modes in such systems is performed after the commands portions of different modes have been independently recognized.…”

Section: Future Directionsmentioning

confidence: 99%

See 1 more Smart Citation

Visual interpretation of hand gestures for human-computer interaction: a review

Pavlović

Sharma

Huang

1997

IEEE Trans. Pattern Anal. Machine Intell.

1,468

646

View full text Add to dashboard Cite

Abstract-The use of hand gestures provides an attractive alternative to cumbersome interface devices for human-computer interaction (HCI). In particular, visual interpretation of hand gestures can help in achieving the ease and naturalness desired for HCI. This has motivated a very active research area concerned with computer vision-based analysis and interpretation of hand gestures. We survey the literature on visual interpretation of hand gestures in the context of its role in HCI. This discussion is organized on the basis of the method used for modeling, analyzing, and recognizing gestures. Important differences in the gesture interpretation approaches arise depending on whether a 3D model of the human hand or an image appearance model of the human hand is used. 3D hand models offer a way of more elaborate modeling of hand gestures but lead to computational hurdles that have not been overcome given the real-time requirements of HCI. Appearance-based models lead to computationally efficient "purposive" approaches that work well under constrained situations but seem to lack the generality desirable for HCI. We also discuss implemented gestural systems as well as other potential applications of vision-based gesture recognition. Although the current progress is encouraging, further theoretical as well as computational advances are needed before gestures can be widely used for HCI. We discuss directions of future research in gesture recognition, including its integration with other natural modes of humancomputer interaction.

show abstract

Section: Localizationmentioning

confidence: 99%

Section: Future Directionsmentioning

confidence: 99%

Visual interpretation of hand gestures for human-computer interaction: a review

Pavlović

Sharma

Huang

1997

IEEE Trans. Pattern Anal. Machine Intell.

1,468

646

View full text Add to dashboard Cite

show abstract

“…These parameters constitute some description of the hand pose or trajectory and depend on the modeling approach used. Among the important problems involved in the analysis are those of hand localization, hand tracking [11], [12], [1] and selection of suitable image features. The computation of model parameters is followed by gesture recognition.…”

Section: Affective Gesture Analysismentioning

confidence: 99%

Emotion Analysis in Man-Machine Interaction Systems

Balomenos

Raouzaiou

Ioannou

et al. 2005

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Facial expression and hand gesture analysis plays a fundamental part in emotionally rich man-machine interaction (MMI) systems, since it employs universally accepted non-verbal cues to estimate the users' emotional state. In this paper, we present a systematic approach to extracting expression related features from image sequences and inferring an emotional state via an intelligent rule-based system. MMI systems can benefit from these concepts by adapting their functionality and presentation with respect to user reactions or by employing agent-based interfaces to deal with specific emotional states, such as frustration or anger.

show abstract

“…For this reason, the concept of Human-Machine Interaction HMI is important because the capabilities of the human users can be extended and the process of learning through the integration of different senses is accelerated (Cole & Mariani, 1995;Sharma et al, 1998;Akay et al, 1998)). Normally, any system that pretends to have a normal interaction must be as natural as possible (Hauptmann & McAvinney, 1993).…”

Section: Development Of a Real-time Gesture Recognition Evaluation Amentioning

confidence: 99%