Pen and touch gestural environment for document editing on interactive tabletops

Abstract

Combined pen and touch input is an interaction paradigm attracting increasing interest both in the research community and recently in industry. In this paper, we illustrate how pen and touch interaction techniques can be leveraged for editing and authoring of presentational documents on digital tabletops. Our system exploits the rich interactional vocabulary afforded by the simultaneous availability of the two modalities to provide gesture-driven document editing functionality as an expert alternative to widgets. For our bimanual gestures, we make use of non-dominant hand postures to set pen modes in which the dominant hand articulates a variety of transactions. We draw an analogy between such modifier postures and modifier keys on a keyboard to construct command shortcuts. Based on this model, we implement a number of common document editing operations, including several page and element manipulations, shape and text input with styling, clipart retrieval and insertion as well as undo/redo. The results of a lab study provide insights as to the strengths and limitations of our approach.

Creativity Support and Multimodal Pen-based Interaction

“…The benefits of using multi-touch and pen input in tandem on large interactive surfaces, such as tabletops are well explored (e.g., [9,20,24,30,45,49]). However, with small-sized interactive surfaces, such as with tablets bimanual touch input faces more constraints.…”

Spatial Querying of Geographical Data with Pen-Input Scopes

“…The latest crop of touchscreen devices and currently dominating interaction paradigms indeed suggest another kind of multimodal input tandem, albeit with a very different role distribution: pen and touch. As demonstrated by extensive prior work on the subject, the combination of those two types of input enable rapid and fluid transitions between coarse touch-based navigational gestures and finer-grained pen interactions to enter content and commands [3,10,12,19]. We consider those properties to be of particular relevance to map-related tasks, with a division of labour consisting of multitouch for panning and scaling the map, while the stylus is used for annotations and commands.…”

Intelligent Ink Annotation Framework that uses User's Intention in Electronic Document Annotation

“…They require users to perform gestures defined by themselves to obtain annotation information in addition to common behaviors of paper-based documents, e.g., a nondominant-hand posture [5] and Pen + Touch interaction [6]. However, this reduces the learnability of such a system.…”