A specification language for direct-manipulation user interfaces

Abstract

A direct-manipulation user interface presents a set of visual representations on a display and a repertoire of manipulations that can be performed on any of them. Such representations might include screen buttons, scroll bars, spreadsheet cells, or flowchart boxes. Interaction techniques of this kind were first seen in interactive graphics systems; they are now proving effective in user interfaces for applications that are not inherently graphical. Although they are often easy to learn and use, these interfaces are also typically difficult to specify and program clearly.Examination of direct-manipulation interfaces reveals that they have a coroutine-like structure and, despite their surface appearance, a peculiar, highly moded dialogue. This paper introduces a specification technique for direct-manipulation interfaces based on these observations. In it, each locus of dialogue is described as a separate object with a single-thread state diagram, which can be suspended and resumed, but retains state. The objects are then combined to define the overall user interface as a set of coroutines, rather than inappropriately as a single highly regular state transition diagram. An inheritance mechanism for the interaction objects is provided to avoid repetitiveness in the specifications. A prototype implementation of a user-interface management system based on this approach is described, and example specifications are given.

An approach integrating two complementary model-based environments for the construction of multimodal interactive applications

“…Over the years, other notations based on finite state automatons have been proposed such as [18] that uses statecharts for describing the behaviour of interactive components but as stated in [14] "… some investigation into extensions of finite state machines to enable them to provide this power led to the realisation that Petri nets already do this". Flownet [53] CPN [32] Z interactor [25] Whizz'ed [27] ATN [47] AME [36] TRIDENT [16] ICO 98 [11] Automata [34] LOTOS [21] Hynet [58] ICO 90 [5] ICON [22] ICO 03 [44] 1970 1980 1990 2000…”

“…ICO 90 [5] Hynet [58] LOTOS [21] Automata [34] ATN [47] CPN [32] ICO 98 [11] TRIDENT [16] AME [37] Whizz'ed [27] Flownet [53] Z interactor [25] ICON [22] ICO 03 [44] 3…”

“…Flownet [53] CPN [32] Z interactor [25] Whizz'ed [27] ATN [47] AME [37] TRIDENT [16] ICO 98 [11] Automata [34] LOTOS [21] Hynet [58] ICO 90 [5] ICON [22] ICO 03 [44] Connections. An input slot of a device can be linked to one or several compatible output slots of other devices by connections, which are represented by wires (see Figure 4).…”

Towards a method for user interface specification

“…The languages which have been examined are Jacob's language for direct manipulation interfaces (Jacob, 1986), Interactive Cooperative Objects (Palanque et aI., 1993), the CNUCE interactor model (Paterno and Faconti, 1992), and Dialogue Specification Notation (Curry and Monk, 1995).…”

Natural Dialogue in Modes other than Natural Language

“…For example, moving the cursor to lie over an object is the command to cause a mode change, because once it is moved, the range of acceptable inputs is reduced and the meaning of each of those inputs is determined. The benefits of ''modelessness'' come from the fact that the mode is always clearly visible (as the location of a cursor in this example), and it has an obvious representation (simply the echo of the same cursor location just used to enter the mode change command) (Jacob, 1986).…”