Web-based Platform for Subtitles Customization and Synchronization in Multi-Screen Scenarios

Montagud, Mario; Boronat, Fernando; González, Juan Antonio Rodríguez; Pastor, Javier

doi:10.1145/3084289.3089913

Cited by 7 publications

(3 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While synchronisation is a complicated goal to achieve due to the state of technology, other improvement proposals tend to dynamic adjustments in the display of subtitles, looking for a re-programming where these subtitles are already synchronised [2]. When re-programming is the starting point, the user experience is improved from the point of view of personalisation and synchronisation, which is usually done dynamically [25], although this synchronisation is not strict, as indicated by the quality parameters [15] [4].…”

Section: Related Workmentioning

confidence: 99%

Deep-Sync: A novel deep learning-based tool for semantic-aware subtitling synchronisation

Martín

González‐Carrasco

Rodríguez-Fernández

et al. 2021

Neural Comput & Applic

View full text Add to dashboard Cite

Subtitles are a key element to make any media content accessible for people who suffer from hearing impairment and for elderly people, but also useful when watching TV in a noisy environment or learning new languages. Most of the time, subtitles are generated manually in advance, building a verbatim and synchronised transcription of the audio. However, in TV live broadcasts, captions are created in real time by a re-speaker with the help of a voice recognition software, which inevitability leads to delays and lack of synchronisation. In this paper, we present Deep-Sync, a tool for the alignment of subtitles with the audio-visual content. The architecture integrates a deep language representation model and a real-time voice recognition software to build a semantic-aware alignment tool that successfully aligns most of the subtitles even when there is no direct correspondence between the re-speaker and the audio content. In order to avoid any kind of censorship, Deep-Sync can be deployed directly on users' TVs causing a small delay to perform the alignment, but avoiding to delay the signal at the broadcaster station. Deep-Sync was compared with other subtitles alignment tool, showing that our proposal is able to improve the synchronisation in all tested cases.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep-Sync: A novel deep learning-based tool for semantic-aware subtitling synchronisation

Martín

González‐Carrasco

Rodríguez-Fernández

et al. 2021

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…Up to date, many subtitling solutions for traditional 2D video content have been proposed, even including support for personalization options (e.g., [30], [31], [32]) and advanced presentation methods for better speaker identification (e.g., [33]). However, subtitling solutions for immersive media are still in their infancy.…”

Section: Subtitling (For the Deaf And Hard-of-hearing Sdh)mentioning

confidence: 99%

Culture 4 all: accessibility-enabled cultural experiences through immersive VR360 content

Montagud

Orero

Matamala

2020

Pers Ubiquit Comput

Self Cite

View full text Add to dashboard Cite

The use of Virtual Reality (VR) technologies and, in particular of VR360 content, can provide great benefits in the society. This particularly applies to the culture and heritage sectors, where venues, goods and events can be digitalized to provide hyper-realistic and engaging experiences, anywhere and anytime. Despite significant advances in the field of VR, there is still an unexplored aspect which is crucial for every service and experience where user interaction is expected: accessibility. This article firstly reviews the needs, challenges and limitations for making VR360 experiences accessible. Based on these facts, an end-to-end platform to efficiently integrate accessibility services within VR360 content is presented. The platform encompasses all steps from media authoring to media consumption, but special attention is given to the accessibility-enabled VR360 player, as it is the end user interaction interface. The presentation modes for the supported access services (like subtitling, audio description and sign language), interaction modalities and personalization features supported by the player are described. To conclude, the availability of newly created and adapted culture-related accessible VR360 content is explored, as a proof of the potential of the contributions work in this sector.

show abstract

“…Furthermore, there are also some proposals that suggest the use of web services for synchronization in batch processes. For example, [22] presents a web-based platform that enables the customization and synchronization of subtitles on both single-and multi-screen scenarios. The platform enables the dynamic customization of the subtitles' format (font family, size, colour, etc.)…”

Section: Literature Reviewmentioning

confidence: 99%

Sub-Sync: Automatic Synchronization of Subtitles in the Broadcasting of True Live programs in Spanish

González‐Carrasco

Puente²,

Ruíz‐Mezcua

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Individuals with sensory impairment (hearing or visual) encounter serious communication barriers within society and the world around them. These barriers hinder the communication process and make access to information an obstacle they must overcome on a daily basis. In this context, one of the most common complaints made by the Television (TV) users with sensory impairment is the lack of synchronism between audio and subtitles in some types of programs. In addition, synchronization remains one of the most significant factors in audience perception of quality in live-originated TV subtitles for the deaf and hard of hearing. This paper introduces the Sub-Sync framework intended for use in automatic synchronization of audiovisual contents and subtitles, taking advantage of current well-known techniques used in symbol sequences alignment. In this particular case, these symbol sequences are the subtitles produced by the broadcaster subtitling system and the word flow generated by an automatic speech recognizing the procedure. The goal of Sub-Sync is to address the lack of synchronism that occurs in the subtitles when produced during the broadcast of live TV programs or other programs that have some improvised parts. Furthermore, it also aims to resolve the problematic interphase of synchronized and unsynchronized parts of mixed type programs. In addition, the framework is able to synchronize the subtitles even when they do not correspond literally to the original audio and/or the audio cannot be completely transcribed by an automatic process. Sub-Sync has been successfully tested in different live broadcasts, including mixed programs, in which the synchronized parts (recorded, scripted) are interspersed with desynchronized (improvised) ones. INDEX TERMS Accessibility, TV broadcasting, algorithm design and analysis, automatic speech recognition.

show abstract

Web-based Platform for Subtitles Customization and Synchronization in Multi-Screen Scenarios

Cited by 7 publications

References 3 publications

Deep-Sync: A novel deep learning-based tool for semantic-aware subtitling synchronisation

Deep-Sync: A novel deep learning-based tool for semantic-aware subtitling synchronisation

Culture 4 all: accessibility-enabled cultural experiences through immersive VR360 content

Sub-Sync: Automatic Synchronization of Subtitles in the Broadcasting of True Live programs in Spanish

Contact Info

Product

Resources

About