Coping With the Challenges of Delivering Multiple Sensorial Media

Saleme, Estêvão Bissoli; Santos, Celso A. S.; Ghinea, Gheorghiţă

doi:10.1109/mmul.2018.2873565

Cited by 19 publications

(32 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Providing an enhanced QoE for mulsemedia applications constitutes an important challenge since ultra-reliable and ultra-responsive connectivity is required. This puts unprecedented pressure on the underlying networks, where reaction times for haptic interactions should be swift 4 .…”

Section: Do I Smell Coffee? the Tale Of A 360º Mulsemedia Experiencementioning

confidence: 99%

“…They proposed a framework named PlaySEM Sensory Effects Renderer (SER) to integrate different applications to varied devices in a straightforward fashion by reusing their mulsemedia renderer that supports MPEG-V, a standard to describe Sensory Effects Metadata (SEM), and provides the necessary protocols to interact with our application. From that perspective and considering the fact that PlaySEM SER has been successfully applied to different scenarios of usage not only for its authors 4 , it catered for our prototype. Thus, the integration of our system reflects the end-user side of the conceptual approach introduced in Figure 1 and follows the tutorial proposed by Saleme et al 14 .…”

Section: º Mulsemedia: User Prototypementioning

confidence: 99%

See 1 more Smart Citation

Do I Smell Coffee? The Tale of a 360° Mulsemedia Experience

et al. 2019

Self Cite

View full text Add to dashboard Cite

Section: Do I Smell Coffee? the Tale Of A 360º Mulsemedia Experiencementioning

confidence: 99%

Section: º Mulsemedia: User Prototypementioning

confidence: 99%

Do I Smell Coffee? The Tale of a 360° Mulsemedia Experience

et al. 2019

Self Cite

View full text Add to dashboard Cite

“…A mulsemedia system entails weaving multiple technologies to connect different entities, distribute the sensory signals, and render sensory effects appropriately Saleme et al [56]. Whilst developing mulsemedia systems, it is crucial to have ways to deliver different sensory content consistently as well as of paramount importance to be aware of the challenges that might arise when delivering mulsemedia [57]. The main motivation behind adding mulsemedia components is to augment the level of immersion and QoE of users [44].…”

Section: Mulsemedia and Qoementioning

confidence: 99%

“…In this emerging field, there have been several explorations on the practicality and possibility of integrating different media types into applications. Thanks to the advent of novel technologies and innovative devices to artificially produce sensory effects along with systems able to deliver this kind of experience to the users [57] the addition of multiple sensory effects has been essential to improving immersion and presence in the user's environment.…”

Section: Introductionmentioning

confidence: 99%

QoE of cross-modally mapped Mulsemedia: an assessment using eye gaze and heart rate

Mesfin

Hussain

Kani‐Zabihi

et al. 2020

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

A great deal of research effort has been put in exploring crossmodal correspondences in the field of cognitive science which refer to the systematic associations frequently made between different sensory modalities (e.g. high pitch is matched with angular shapes). However, the possibilities cross-modality opens in the digital world have been relatively unexplored. Therefore, we consider that studying the plasticity and the effects of crossmodal correspondences in a mulsemedia setup can bring novel insights about improving the human-computer dialogue and experience. Mulsemedia refers to the combination of three or more senses to create immersive experiences. In our experiments, users were shown six video clips associated with certain visual features based on color, brightness, and shape. We examined if the pairing with crossmodal matching sound and the corresponding auto-generated haptic effect, and smell would lead to an enhanced user QoE. For this, we used an eye-tracking device as well as a heart rate monitor wristband to capture users' eye gaze and heart rate whilst they were experiencing mulsemedia. After each video clip, we asked the users to complete an on-screen questionnaire with a set of questions related to smell, sound and haptic effects targeting their enjoyment and perception of the experiment. Accordingly, the eye gaze and heart rate results showed significant influence of the cross-modally mapped multisensorial effects on the users' QoE. Our results highlight that when the olfactory content is crossmodally congruent with the visual content, the visual attention of the users seems shifted towards the correspondent visual feature. Crosmodally matched media is also shown to result in an enhanced QoE compared to a video only condition.

show abstract

“…Table 2 presents the instantiation of the ontology. Bearing in mind that most of the mulsemedia systems presented in [16,18,19,[28][29][30]34] are smaller, the successful instantiation of the whole ontology with this hypothetical scenario presented in Table 2 gives us a broad sense of what is possible to represent with this reference ontology. Moreover, this instantiation cues for appropriateness of the proposed ontology as a reference model for this domain.…”

Section: Directly Transmitted Mediummentioning

confidence: 99%

Towards a reference ontology on mulsemedia systems

Saleme

Santos

Falbo

et al. 2018

Proceedings of the 10th International Conference on Management of Digital EcoSystems

Self Cite

View full text Add to dashboard Cite

The use of multiple senses in interactive applications has become increasingly feasible due to the upsurge of commercial, off-the-shelf devices to produce sensory effects. Creating Multiple Sensorial Media (MulSeMedia) immersive systems requires understanding their digital ecosystem. Mulsemedia systems encompass a set of applications, and devices of different types assembled to communicate or express feelings from the virtual world to the real world. Despite existing standards, tools, and recent research devoted to them, there is still a lack of formal and explicit representation of what mulsemedia is. Misconceptions could eventually lead to the construction of solutions that might not take into account reuse, integration, standardization, among other design features. In this paper, we propose to establish a common conceptualization about mulsemedia systems through a reference ontology, named MulseOnto, covering their main notions. To evaluate it, we applied ontology verification and validation techniques, including assessment by humans and a data-driven approach. The results showed that MulseOnto can be used as a consensual conceptual model for exploring the knowledge about the whole chain of mulsemedia systems. CCS CONCEPTS • General and reference → Reference works; • Information systems → Multimedia information systems; • Human-centered computing → Ubiquitous and mobile computing theory, concepts and paradigms;

show abstract

Coping With the Challenges of Delivering Multiple Sensorial Media

Cited by 19 publications

References 15 publications

Do I Smell Coffee? The Tale of a 360° Mulsemedia Experience

Do I Smell Coffee? The Tale of a 360° Mulsemedia Experience

QoE of cross-modally mapped Mulsemedia: an assessment using eye gaze and heart rate

Towards a reference ontology on mulsemedia systems

Contact Info

Product

Resources

About