A Framework to Develop Adaptive Multimodal Dialog Systems for Android-Based Mobile Devices

Abstract

Abstract.Mobile devices programming has emerged as a new trend in software development. The main developers of operating systems for such devices have provided APIs for developers to implement their own applications, including different solutions for developing voice control. Android, the most popular alternative among developers, offers libraries to build interfaces including different resources for graphical layouts as well as speech recognition and text-to-speech synthesis. Despite the usefulness of such classes, there are no strategies defined for multimodal interface development for Android systems, and developers create ad-hoc solutions that make apps costly to implement and difficult to compare and maintain. In this paper we propose a framework to facilitate the software engineering life cycle for multimodal interfaces in Android. Our proposal integrates the facilities of the Android API in a modular architecture that emphasizes interaction management and context-awareness to build sophisticated, robust and maintainable applications.

Research on the Parametric Data Acquisition Technology for Rural Real Estate Fieldwork

“…According to the interaction event on Android devices [10], the interactive annotation function is designed. Respond to self-defined click event, long press event, control click event as well as trigger tagging function dialog box by realizing OnClickListener interface [11]. As shown in the Figure 8: ① Trigger dimension input dialog box to record dimension by house contour line pickup algorithm with drawText() method in the Canvas used to draw characters in the middle on one side of the line segment for completing annotation: ② Trigger self-defined attribute dialog box by planar graph pickup algorithm and select input attribute information in Spinner control; ③ Enable the camera function dialog box to record the current general layout of house with photo recorded into local folders.…”

Incorporating android conversational agents in m‐learning apps

“…As will be described in the following section, its capabilities to support comfort and user awareness, rich dialogue model and intent adaptation make it a very relevant choice for the development of mobile learning apps. Figure 1 shows our framework for the development of adaptive multimodal conversational agents for mobile devices that was presented in (Griol and Molina, 2014). A multimodal conversational agent providing spoken interaction integrates five main tasks to deal with user's spoken utterances: automatic speech recognition, natural language understanding, dialogue management, natural language generation and text-to-speech synthesis (TTS).…”

“…In our previous work (Griol and Molina, 2014), we proposed a general-purpose framework to develop conversational interfaces for mobile devices. It provides a modular and scalable architecture to develop such systems efficiently for Android-based mobile devices.…”