MPML3D: A Reactive Framework for the Multimodal Presentation Markup Language

Abstract

MPML3D is our first candidate of the next generation of authoring languages aimed at supporting digital content creators in providing highly appealing and highly interactive content with little effort. The language is based on our previously developed family of Multimodal Presentation Markup Languages (MPML) that broadly followed the "sequential" and "parallel" tagging structure scheme for generating presynchronized presentations featuring life-like characters and interactions with the user. The new markup language MPML3D deviates from this design framework and proposes a reactive model instead, which is apt to handle interaction-rich scenarios with highly realistic 3D characters. Interaction in previous versions of MPML could be handled only at the cost of considerable scripting effort due to branching. By contrast, MPML3D advocates a reactive model that allows perceptions of other characters or the user interfere with the presentation flow at any time, and thus facilitates natural and unrestricted interaction. MPML3D is designed as a powerful and flexible language that is easy-to-use by non-experts, but it is also extensible as it allows content creators to add functionality such as a narrative model by using popular scripting languages.

Automated generation of non-verbal behavior for virtual embodied characters

“…The Multimodal Presentation Markup Language is used to control the behavior of our 3D agents [7]. We choose a modular pipelined architecture to support future extensions.…”

T2D: Generating Dialogues Between Virtual Agents Automatically from Text

“…Mapper: Module that maps RST structures to DialogueNet structures (these are a specific subclass of RST structures that represent dialogue); 3. Presenter: A Module for translating DialogueNet structures to the Multimodal Presentation Markup Language (MPML3D) format [13]. MPML3D script specifies multi-modal dialogue performed by two 3D agents.…”

“…Finally, the purpose of the Presenter Module is to translate the DialogueNet structure into MPML3D [13], our Multimodal Presentation Markup Language for highly realistic 3D agents (see Fig. 2).…”

Attentive Presentation Agents

“…The Multimodal Presentation Markup Language for 3D agents (MPML3D) provides a easy-to-use XML scripting language to define both the content of the presentation as well as the animations of the agents [7,11]. The main concept of MPML3D are actions: every (speech) utterance of an agent, every gesture and mimic expression, etc.…”