Enhancing the quality of service of mobile video technology by increasing multimodal synergy

Abstract

Bandwidth is still a limiting factor for the Quality of Service (QoS) of mobile communication applications. In particular, for Voice over IP the QoS is not yet as good as for common, wellengineered, public-switched telephone networks. Multisensory communication has been identified as a possibility to moderate this limitation. One of the strengths of mobile video technology lies in its combination of visual and auditory modalities. However, one of the most salient features of mobile video applications is its small screen size. To test the potential of multimodal synergy for mobile devices, we assessed to what extent small screens affect multimodal synergy. This potential was assessed in an experiment with 54 participants, who conducted a standardised video-listening test for three talking-heads videos with a signal-tonoise ratio of-9 dB. The videos were presented on three different screen sizes, whilst keeping the video and auditory signals equal. Compared to a ground truth based on 359 participants, intelligibility was found to be significantly higher when using a large screen than when using a small screen. This indicates that mobile video technology has the potential for a significant multimodal synergy to which screen size is a substantial constraint. To optimally benefit from their multimodal potential, we offer suggestions on how to increase the effective screen size for small screen (e.g. mobile) devices and applications through elaborating the most relevant (visual) features. We conclude that knowledge about human sensory processing can alleviate the identified constraint and maximise the potential QoS of mobile video technology.

Handling sensory disabilities in a smart society

“…The studies underline the importance of personal preference for the favorite mode of interaction, even though we can discern a general preference for the multimodal mode, with see suggested to be the most important aspect (cf. [59]) in the initial study.…”

“…For instance, older adults, who usually experience some degree of vision loss, could benefit from multimodal devices [64]. Following these two lines of thought, we propose that using an intelligent agent in the form of a social robot as a butler managing the IE, and interacting with the user through different modalities is helpful for accessibility [59].…”

“One for All, All for One”. A First Step Towards Universal Access with a Social Robot

“…Nevertheless, this study already highlighted several interesting aspects such as a feeling of unfairness with the hear mode, differences in preference of interaction, and a general preference for the multimodal mode, with see suggested to be the most important aspect (cf. [23]).…”

Interactive Multimodal Television Media Adaptive Visual Communication Based on Clustering Algorithm

“…e Internet is an important part of the daily life of the audience. It is this change in lifestyle that has gradually changed the way of information dissemination [4]. e original single-modal, linear form of communication has changed to a multimodal, nonlinear form and digital technology.…”