Multimodal Generic Framework for Multimedia Documents Adaptation

Abstract

Today, people are increasingly capable of creating and sharing documents (which generally are multimedia oriented) via the internet. These multimedia documents can be accessed at anytime and anywhere (city, home, etc.) on a wide variety of devices, such as laptops, tablets and smartphones. The heterogeneity of devices and user preferences has raised a serious issue for multimedia contents adaptation. Our research focuses on multimedia documents adaptation with a strong focus on interaction with users and exploration of multimodality. We propose a multimodal framework for adapting multimedia documents based on a distributed implementation of W3C's Multimodal Architecture and Interfaces applied to ubiquitous computing. The core of our proposed architecture is the presence of a smart interaction manager that accepts context related information from sensors in the environment as well as from other sources, including information available on the web and multimodal user inputs. The interaction manager integrates and reasons over this information to predict the user's situation and service use. A key to realizing this framework is the use of an ontology that undergirds the communication and representation, and the use of the cloud to insure the service continuity on heterogeneous mobile devices. Smart city is assumed as the reference scenario.

A Multi-Viewpoint Approach For Semantic Multimedia Documents Adaptation

“…In [3,13], the authors have proposed a mechanism allowing dynamic generation of user conditions by including multi modal technologies. In [14], the authors have introduced a context-based multimedia adaptation approach that combines multi-agent systems with semantic web services to predict the current context of the user.…”

“…However, the system requires more work since it is not implemented and evaluated. However, the systems in [3,13,14] require more work since it is not implemented and evaluated.…”

A goal-driven software product line approach for evolving multi-agent systems in the Internet of Things

“…The AppE entails the creation and configuration of one of the family members. CPS and IoT Systems are composed of a heterogeneous set of devices and network technologies [45]. So, the identification and management of this variability is performed at the DomE level.…”