Vision-based user interfaces: methods and applications

Throughout the history of CAD, a variety of interfaces have been employed with the aim of supporting designers' construction and manipulation of digital models. This paper explores the potential of eye tracking as a CAD interface, and presents a prototype that uses collected gaze data to support shape exploration in a two-dimensional vectorbased sketch editor. The eye tracking interface uses the visual interactions of users to identify interpretations of constructed shapes, according to recognised parts identified using methods from shape grammar research. It can therefore be argued that it supports construction and manipulation of shapes according to user intent. The prototype has been evaluated in a user study where design students carried out controlled shape exploration exercises which involved interpretation and manipulation of shapes. In the paper, key concepts of eye tracking methodology are introduced; the methods used to implement the eye tracking interface are described; and the user study and its outcomes are reported. The results are positive and indicate the potential for eye tracking as an interface for supporting shape exploration in CAD. Research Highlights• Explores the potential of eye tracking as an interface for computer aided-design• Describes methods for inferring user intent with respect to digital shape manipulation, based on gaze data • Presents a prototype eye tracking interface that supports dynamic shape interpretation in a vector-based sketch editor • Reports the outcomes of a user study that evaluates how successful the eye tracking interface is at inferring users' intent with respect to shape manipulation

show abstract

“…[31]. When interacting with such objects user input is typically preceded by movement of gaze to areas of interest.…”

Section: Figure 3 Example Of Eye Tracking Datamentioning

confidence: 99%

Evaluating an eye tracking interface for a two-dimensional sketch editor

Jowers

Prats

McKay

et al. 2013

Computer-Aided Design

View full text Add to dashboard Cite

show abstract

“…Identifying is the action of interpreting the event or the phenomenon; for example, the user is executing a gesture with such (geometric) features. Tracking is the result of observing the evolution of the phenomenon, computing its actual state and relating it with the past states; for example, the user has drawn a shape recognized by the system [13].…”

Section: Gesture Vs Shape Recognitionmentioning

confidence: 99%

Gestures, Shapes and Multitouch Interaction

Celentano

Minuto

2008

2008 19th International Conference on Database and Expert Systems Applications

View full text Add to dashboard Cite

Abstract-We discuss issues related to the design of a multitouch gesture sensing environment, allowing the user to execute both independent and coordinated gestures. We discuss different approaches, comparing frontal vs. back projection devices and gesture tracking vs. shape recognition. To compare the approaches we introduce a simple gesture language for drawing diagrams. A test multitouch device built around FTIR technology is illustrated; a vision system, driven by a visual dataflow programming environment, interprets the user gestures and classifies them into a set of predefined patterns, corresponding to language commands.has received great attention in last years, due to progress in research about interaction devices and to new applications, mainly in the Web2.0 area. Ubiquitous computing and ambient intelligence compel us to rethink our relations with information devices, trying to surpass the traditional "window, icon, menu, pointing device" (WIMP) style of interaction with multimodal and distributed interfaces.The WIMP paradigm is, however, so well rooted that many efforts to find more natural interaction styles through free gesture interpretation are still based on the model of a pointer moving in a limited area, commanding actions by "clicking" on objects representing programs and documents. Indeed, the 2D desktop layout doesn't leave too much space to freely propose "real world" gestures, possible only in an immersive environment and with constraints.In this paper we argument about the choice of a gesture sensing device and the design of a gesture language for it. We discuss the pros and cons of different interaction styles, and their impact on the gesture structure and interpretation. We define a drawing-style gesture language suited for a multitouch device based on the Frustrated Total Internal Reflection (FTIR) technololgy proposed by Han [4], allowing users to interact with real multitouch, independent gestures. The gesture language is targeted to drawing simple graphs such as the ones used in conceptual and mental maps. The operations permitted are: create an object (a node), move an object, connect two or more objects, write text labels, delete objects.We have built a prototype system to experiment our proposal. Gesture are captured and interpreted by a vision system built around a dataflow programming environment; gestures are classified into a set of predefined patterns corresponding to the language commands. Having multiple pointers allows the user to execute both parallel independent gestures (e.g., to select many objects at the same time), or coordinated gestures (e.g., to connect two objects).

show abstract

“…Extensive surveys have been published in several HCI fields such as face detection (Yang et al, 2002), face recognition (Zhao et al, 2003), facial expression analysis (Fasel and Luettin, 2003), gesture recognition (Pavlovic et al, 1997, Mitra andAcharya, 2007), human motion analysis (Gavrila, 1999, Aggarwal and Cai, 1999, Wang et al, 2003. A survey presenting the use of vision in HCI, particularly in the area of head tracking can be found in work (Porta, 2002, Kisacanin, 2005. The authors of work (Jaimes and Sebe, 2007) give an overview of multimodal human machine interfaces.…”

Section: Current Researchmentioning

confidence: 99%

Adaptive Real-Time Image Processing for Human-Computer Interaction

Kwolek¹

2008

Human Computer Interaction: New Developments

View full text Add to dashboard Cite

Vision-based user interfaces: methods and applications

Cited by 77 publications

References 118 publications

Evaluating an eye tracking interface for a two-dimensional sketch editor

Evaluating an eye tracking interface for a two-dimensional sketch editor

Gestures, Shapes and Multitouch Interaction

Adaptive Real-Time Image Processing for Human-Computer Interaction

Contact Info

Product

Resources

About