Natural Dialogue in Modes other than Natural Language

Abstract

Each command or transaction in a modern graphical user interface exists as a nearly independent utterance, unconnected to previous and future ones from the same user. This is unlike real human communication, where each utterance draws on previous ones for its meaning. While some natural language human-computer interfaces attempt to include such characteristics of human dialogue, they have been notably absent from graphical user interfaces. Our goal is to connect these properties of human dialogue to direct manipulation or graphical styles of user-computer interaction. This chapter describes our approach to incorporating some of the properties of natural dialogue (such as conversational flow, discourse, focus) into "other" (that is, not natural language) modes of human-computer communication. It also describes our research on building a natural user-computer dialogue based on a user's eye movements. That work shows that starting from natural, rather than trained, eye movements results in a more natural dialogue with the user.

COLLAGEN: A Collaboration Manager for Software Interface Agents

“…Although this distinction is not highlighted in our current scenarios, it will become important in future extensions. Cohen (1992) and Jacob (1995), among others, have explored discourserelated extensions to direct manipulation that incorporate anaphora and make previous context directly available. However, most work on applying human discourse principles to human-computer interaction, e.g., (Lambert and Carberry, 1991;Yanklovich, 1994), have assumed that natural language understanding will be applied to the user's utterances In Moore et al's work (Lemaire and Moore, 1994;Moore and Swartout, 1990), which focuses on explanation dialogues, users are presented with a full textual history of their interaction with the system, from which they may select any phrase as the context for a further query.…”

New Human-Computer Interaction Techniques

“…Another direction in our research is the notion of dialogue interaction techniques [6,8]. In a direct manipulation or graphical interface, each command or brief transaction exists as a nearly independent utterance, unconnected to previous and future ones from the same user.…”