Towards a formalization of multi-touch gestures

Abstract

Multi-touch is a technology which offers new styles of interaction compared to traditional input devices like keyboard and mouse. Users can quickly manipulate objects or execute commands by means of their fingers and hands. Current multi-touch frameworks offer a set of standard gestures that are easy to use when developing an application. In contrast, defining new gestures requires a lot of work involving low-level recognition of touch data. To address this problem, we contribute a discussion of strategies towards a formalization of gestural interaction on multi-touch surfaces. A test environment is presented, showing the applicability and benefit within multi-touch frameworks.

Synchronous gesture manipulation for collaboration and coordination of co-located Business Process Modelling

“…One type of technology that supports touch input is the multitouch surface. The multi-touch is a collaborative technology that allows co-located individuals to collaborate, while interacting together on one surface simultaneously using their fingers as a means of touch input instead of a keyboard or mouse (Morris et al, 2006;Hornecker et al, 2008;Kammer, Wojdziak, Keck, Groh & Taranko, 2010;Hunter & Maes, 2008). Using this type of input is a direct way of interacting with the screen and it has been reported to feel like a more natural approach.…”

“…Using a multi-touch surface in a collaborative environment poses many benefits, such as being able to see other participants' body language, participants' pointing to objects on the surface and discussing the work at hand together (Clifton et al, 2011). Collaborative drawing and modelling can also be successfully achieved on a multi-touch surface (Morris et al, 2006;Hornecker et al, 2008;Kammer et al, 2010;Hunter & Maes, 2008;Sams, Wesson & Vogts, 2011).…”

“…The PSSUQ consists of three areas, namely: Information Quality (InfoQual), Interface Quality (IntQual) and System Usefulness (SysUse). Collaboration and satisfaction with gesture manipulation was measured by metrics identified in literature (Hornecker et al, 2008;Kammer et al, 2010;Hunter & Maes, 2008;Sams et al, 2011;Tse et al, 2007). The second and third categories of satisfaction evaluated related to satisfaction with the collaboration and gesture manipulation features of the technology respectively.…”

Gestural query specification

“…Studies on multitouch interfaces [18,54] have demonstrated the superiority of gestural interaction over mouse-driven WIMP paradigms. Kammer et al [33] formalize gestures according to the semantics, pragmatics and syntactics of interaction, while Yee [59] discusses concerns regarding direct and indirect gestural interactions. Visual analytics systems such as Tableau [51] and TaP [21] map interactions and gestures from the UI layer to a set of database query templates, without considering the contents of the database itself.…”

gestUI: a model-driven method for including gesture-based interaction in user interfaces

“…Kammer et al [85] describe GeForMT (Gesture Formalization for MultiTouch) which is defined using semiotic with three components and their scope: syntactic (symbols), semantics (meaning) and pragmatics (interpretation). This language permits the representation of multi-touch gestures.…”