GestUI: A Model-driven Method and Tool for Including Gesture-based Interaction in User Interfaces

Abstract

Abstract. Among the technological advances in touch-based devices, gesturebased interaction have become a prevalent feature in many application domains. Information systems are starting to explore this type of interaction. As a result, gesture specifications are now being hard-coded by developers at the source code level that hinders their reusability and portability. Similarly, defining new gestures that reflect user requirements is a complex process. This paper describes a model-driven approach to include gesture-based interaction in desktop information systems. It incorporates a tool prototype that captures usersketched multi-stroke gestures and transforms them into a model by automatically generating the gesture catalogue for gesture-based interaction technologies and gesture-based user interface source codes. We demonstrated our approach in several applications ranging from case tools to form-based information systems.

pEncode: A Tool for Visualizing Pen Signal Encodings in Real-Time

“…Over the last years, the popularity of digital pen input and touch-based interaction significantly increased. They are used in various tasks such as gesture-based user interfaces (UIs) [28,14,4,8,15,17,2,9], handwriting recognition [11,10], signature verification [26], cognitive state assessments [22,18,7,25,23,19], and multimodal learning analysis [3,16]. Recent studies have also explored the interpretability of such systems [19,12,29,18,5,13].…”