A Discourse and Dialogue Infrastructure for Industrial Dissemination

Abstract

Abstract. We think that modern speech dialogue systems need a prior usability analysis to identify the requirements for industrial applications. In addition, work from the area of the Semantic Web should be integrated. These requirements can then be met by multimodal semantic processing, semantic navigation, interactive semantic mediation, user adaptation/personalisation, interactive service composition, and semantic output representation which we will explain in this paper. We will also describe the discourse and dialogue infrastructure these components develop and provide two examples of disseminated industrial prototypes.

RadSpeech's mobile dialogue system for radiologists

“…Such components include a speech recogniser (ASR) and a speech synthesis (TTS) module. On the client, e.g., an iPad, only a slim application is needed to encode the speech and gesture input and receive the speech synthesis to be played ( Figure 2, details are explained in [2]). New clients can be added easily by, e.g., downloading an iPad app.…”

“…Although it is widely reductive to put it this way, a senior radiologist has three main goals: (1) access the images and image (region) annotations, (2) complete them, and (3) refine existing annotations. We argued that these tasks can best be fulfilled while using a multimodal dialogue system, and first we experimented with a large touchscreen installation [1,2]. Since the results with speech interaction were very promising, we tried to opt for the mobile context.…”

ERmed – Towards Medical Multimodal Cyber-Physical Environments

“…The user is then able to employ multiple modalities, like speech and gestures, to interact with the presented multimedia material in a multimodal way. In project THESEUS [18] we extended SmartWeb's multimodal dialogue system towards integrating the aforementioned semantic web technologies for the Web 3.0 and industrial applications. THESEUS was the German flagship project on the Internet of Services, where the user can delegate complex tasks to dynamically composed semantic web services by utilizing multimodal interaction combining speech and multi-touch input on advanced smartphones.…”

“…For the industrial application tasks, we rely on a multimodal discourse and dialogue infrastructure [18] and implemented smartphone gestures interactions in combination with speech (further explained in [20]). New AR interaction possibilities are integrated as follows: essentially, the dialogue context and previous utterances define what the user sees in the mobile HMD.…”

VoiceXML for Pervasive Environments

“…ODP supports standards such as MRCP/SIP, EMMA and SSML and GUI frameworks such as Ajax, Flash/Flex or JavaFX as well as novel modalities (for example multitouch, gesture recognition or virtual characters) (http://www.semvox.de/ en/menu-home.html). SemVox also provides its own workbench for rapid application development (Sonntag, Sonnenberg, Nesselrath, & Herzog 2009). Applications are written in an proprietary XML format.…”