Toward Content-Driven Intelligent Authoring of Mulsemedia Applications

Abreu, Raphael; Mattos, Douglas Paulo de; Santos, Joel dos; Ghinea, Gheorghiţă; Muchaluat-Saade, Débora C.

doi:10.1109/mmul.2020.3011383

Cited by 14 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Since the tool provides a temporal view in the authoring GUI (Graphical User Interface), users do not need to have programming skills to define a mulsemedia application using the STEVE 2.0 environment. It also gives authors feedback about temporal synchronization inconsistencies and provides automatic extraction of sensory effects from audiovisual content analyses by applying machine learning-based methods [1]. STEVE 2.0 provides NCL (Nested Context Language) 2 as a final format that can be run in the Ginga-NCL player, an extension of the Brazilian Digital TV middleware.…”

Section: Related Workmentioning

confidence: 99%

“…The requirements related to 360°content, layout and scenes were based on requirements proposed in studies such as [6], [18]. In respect of the requirement to use an interchangeable format to import and export projects, in addition to relationship synchronization tools, we were inspired by [1], [18]. Regarding the support for sensory effects, we got some ideas from [4], [6].…”

Section: Proposal For Immersive Authoring Of 360°m Ulsemediamentioning

confidence: 99%

“…Based on the videogame development industry, which is a current environment for VR content creation [23], the work of [18] recommends the use of VR to produce and place objects within the content in order to improve production and provide greater fidelity in the final product. For this particular type of technology, game engines, such as Unity 1 , offer good production support for VR content, as well as support for standard media, such as image, video and audio, user interaction and even integration with sensory effects [20], [21].…”

Section: Introductionmentioning

confidence: 99%

“…Additionally, we can better interpret content offered by an application combining different senses [23]. We have proposals for authoring sensory effects and interactive events synchronized with traditional multimedia content, such as MultiSEM [10] and STEVE [1]. However, there are fewer applications and methodologies for this purpose targeting VR and 360°content.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Immersive Authoring of 360 Degree Interactive Applications

et al. 2022

Self Cite

View full text Add to dashboard Cite

Although there are proposals in the literature for authoring mulsemedia (multiple sensorial media) applications with 2D content, there are no suitable solutions when it comes to 360°content. Moreover, little consensus on 360°mulsemedia authoring methodology exists. Aiming at filling this gap, we propose the concept of immersive authoring of 360°multisensory applications. Our proposal comprises an immersive 360°authoring environment to bring the author closer to the final user presentation environment. We implemented our proposal in AMUSEVR, a virtual-reality (VR) environment for authoring 360°m ulsemedia applications. We see it as an alternative or a possible complement to available 2D mulsemedia authoring tools. AMUSEVR provides creation and editing of interactive multiple sensorial media scenes by directly arranging objects in a 3D space using VR technology. Also, the tool allows users to run their applications through AMUSEVR viewer mode. We used the Goal Question Metric (GQM) approach to plan our tests and a group of users evaluated the tool with the SUS and UEQ questionnaires, obtaining a SUS score of 82.25 and an excellent UEQ benchmark, which are very promising results.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Proposal For Immersive Authoring Of 360°m Ulsemediamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Immersive Authoring of 360 Degree Interactive Applications

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Since the tool provides a temporal view as the authoring GUI approach, users do not need to have programming skills to define a mulsemedia application using STEVE 2.0 environment. Moreover, STEVE 2.0 provides automatic extraction of sensory effects from audiovisual content analyses by applying Machine Learning-based methods [1]. However, STEVE 2.0 does not provide final formats that can be run in mulsemedia players yet, since the tool implementation is a work in progress.…”

Section: Mulsemedia Authoring Toolsmentioning

confidence: 99%

Beyond Multimedia Authoring

2021

Self Cite

View full text Add to dashboard Cite

The mulsemedia (Multiple Sensorial Media (MulSeMedia)) concept has been explored to provide users with new sensations using other senses beyond sight and hearing. The demand for producing such applications has motivated various studies in the mulsemedia authoring phase. To encourage researchers to explore new solutions for enhancing the mulsemedia authoring, this survey article reviews several mulsemedia authoring tools and proposals for representing sensory effects and their characteristics. The article also outlines a set of desirable features for mulsemedia authoring tools. Additionally, a multimedia background is discussed to support the proposed study in the mulsemedia field. Open challenges and future directions regarding the mulsemedia authoring phase are also discussed.

show abstract