Towards Sketch-based User Interaction with Integrated Software Development Environments

Abstract

Powerful software tools, such as software development environments, often have complex graphical user interfaces (GUIs) that are not intuitive to handle, especially when performing complex, multistep operations. We hypothesize that sketching could be a more intuitive way of expressing user intentions than navigating nested menus or memorizing keyboard shortcuts to accomplish complex operations. Enabling this vision requires software capable of both allowing the user to sketch anywhere on a GUI, and interpreting those sketches as specific commands to be performed within the integrated development environment (IDE). In this paper, we report on preliminary results of an elicitation study performed to gather insights into how developers would use a sketch-based interface. CCS CONCEPTS• Human-centered computing → Interaction techniques; Empirical studies in interaction design; Touch screens; • Software and its engineering → Integrated and visual development environments.

Automated Workflow for Usability Audits in the PHR Realm

“…In the literature, it has been hypothesised to use sketches to learn the usage intentions in the systems, avoiding the user to access nested menus or memorise keyboard shortcuts. For this purpose, software has been used that allows the user to draw sketches on the GUI [ 103 ]. Another alternative is Guided Finger-Aware Shortcuts which aims to reduce the gap between graphical input and the activation of a shortcut.…”