A computer-vision-assisted system for Videodescription scripting

Gagnon, Langis; Chapdelaine, Claude; Byrns, D.; Foucher, Samuel; Héritier, Maguelonne; Gupta, Varun

doi:10.1109/cvprw.2010.5543575

Cited by 18 publications

(12 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the potential benefit of DVS for computer vision, it has not been used so far apart from [25,42] who study how to automate DVS production. We believe the main reason for this is that it is not available in the text format, i.e.…”

Section: The Movie Description Datasetmentioning

confidence: 99%

A dataset for Movie Description

Rohrbach

Tandon

et al. 2015

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

356

258

View full text Add to dashboard Cite

Descriptive video service (DVS) provides linguistic descriptions of movies and allows visually impaired people to follow a movie along with their peers. Such descriptions are by design mainly visual and thus naturally form an interesting data source for computer vision and computational linguistics. In this work we propose a novel dataset which contains transcribed DVS, which is temporally aligned to full length HD movies. In addition we also collected the aligned movie scripts which have been used in prior work and compare the two different sources of descriptions. In total the Movie Description dataset contains a parallel corpus of over 54,000 sentences and video snippets from 72 HD movies. We characterize the dataset by benchmarking different approaches for generating video descriptions. Comparing DVS to scripts, we find that DVS is far more visual and describes precisely what is shown rather than what should happen according to the scripts created prior to movie production.

show abstract

Section: The Movie Description Datasetmentioning

confidence: 99%

A dataset for Movie Description

Rohrbach

Tandon

et al. 2015

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

356

258

View full text Add to dashboard Cite

show abstract

“…3PlayMedia's post-production audio description tool [5] and Gagnon et al's audio description tool [18], both let authors provide text descriptions on videos, synthesizing text-to-speech for playback of the descriptions. Gagnon et al's tool also provides authors with timeline-based visualisations tailored to the production of cinematic audio descriptions including recognition of scenes, characters, and important locations [18]. While prior work suggests that people prefer human-narrated audio descriptions to speech-to-text audio descriptions when available [15,28], the aforementioned systems do not support narrated audio descriptions.…”

Section: Describing Videosmentioning

confidence: 99%

Rescribe

Pavel

Reyes

Bigham

2020

Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

Audio descriptions make videos accessible to those who cannot see them by describing visual content in audio. Producing audio descriptions is challenging due to the synchronous nature of the audio description that must fit into gaps of other video content. An experienced audio description author will produce content that fits narration necessary to understand, enjoy, or experience the video content into the time available. This can be especially tricky for novices to do well. In this paper, we introduce a tool, Rescribe, that helps authors create and refine their audio descriptions. Using Rescribe, authors first create a draft of all the content they would like to include in the audio description. Rescribe then uses a dynamic programming approach to optimize between the length of the audio description, available automatic shortening approaches, and source track lengthening approaches. Authors can iteratively visualize and refine the audio descriptions produced by Rescribe, working in concert with the tool. We evaluate the effectiveness of Rescribe through interviews with blind and visually impaired audio description users who give feedback on Rescribe results. In addition, we invite novice users to create audio descriptions with Rescribe and another tool, finding that users produce audio descriptions with fewer placement errors using Rescribe.

show abstract

“…This process, however, could be facilitated by the use of semi-automated visual recognition techniques, which have been developed in different contexts (such as surveillance and video database indexing). An early example is VDManager [7], a VD editing software tool, which uses speech recognition as well as key-places and key-faces visual recognition.…”

Section: Conclusion and New Frontiersmentioning

confidence: 99%

(Computer) vision without sight

2012

View full text Add to dashboard Cite

Computer vision holds great promise for helping persons with blindness or visual impairments (VI) to interpret and explore the visual world. To this end, it is worthwhile to assess the situation critically by understanding the actual needs of the VI population and which of these needs might be addressed by computer vision. This article reviews the types of assistive technology application areas that have already been developed for VI, and the possible roles that computer vision can play in facilitating these applications. We discuss how appropriate user interfaces are designed to translate the output of computer vision algorithms into information that the user can quickly and safely act upon, and how system-level characteristics affect the overall usability of an assistive technology. Finally, we conclude by highlighting a few novel and intriguing areas of application of computer vision to assistive technology.

show abstract

A computer-vision-assisted system for Videodescription scripting

Cited by 18 publications

References 11 publications

A dataset for Movie Description

A dataset for Movie Description

Rescribe

(Computer) vision without sight

Contact Info

Product

Resources

About