Subtitle-free Movie to Script Alignment

Sankar, K. Pramod; Jawahar, C. V.; Zisserman, Andrew

doi:10.5244/c.23.121

Cited by 31 publications

(14 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are some scenarios where only transcripts are used as the form of supervision. [22] proposes a method to align scripts to videos in the absence of subtitles. [21] jointly model co-reference resolution and character identification using only scripts.…”

Section: Related Workmentioning

confidence: 99%

Naming TV characters by watching and analyzing dialogs

Haurilet

Tapaswi

Al-Halah

et al. 2016

2016 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Person identification in TV series has been a popular research topic over the last decade. In this area, most approaches either use manually annotated data or extract character supervision from a combination of subtitles and transcripts. However, both approaches have key drawbacks that hinder application of these methods at a large scale -manual annotation is expensive and transcripts are often hard to obtain. We investigate the topic of automatically labeling all character appearances in TV series using information obtained solely from subtitles. This task is extremely difficult as the dialogs between characters provide very sparse and weakly supervised data.We address these challenges by exploiting recent advances in face descriptors and Multiple Instance Learning methods. We propose methods to create MIL bags and evaluate and discuss several MIL techniques. The best combination achieves an average precision over 80% on three diverse TV series. We demonstrate that only using subtitles provides good results on identifying characters in TV series and wish to encourage the community towards this problem.

show abstract

Section: Related Workmentioning

confidence: 99%

Naming TV characters by watching and analyzing dialogs

Haurilet

Tapaswi

Al-Halah

et al. 2016

2016 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

show abstract

“…When subtitle information is either not available or there are scenes without dialogue, other methods need to be used to provide alignment information. Sankar et al [28] use a combination of location recognition, facial recognition and speech-to-text to align scripts when subtitles are not available. However, the approach requires manual labelling of characters and location information was hard-coded to repetitive use of stock footage.…”

Section: Related Workmentioning

confidence: 99%

Friendly Faces: Weakly Supervised Character Identification

Marter

Hadfield

Bowden

2015

Face and Facial Expression Recognition From Real World Videos

View full text Add to dashboard Cite

Abstract. This paper demonstrates a novel method for automatically discovering and recognising characters in video without any labelled examples or user intervention. Instead weak supervision is obtained via a rough script-to-subtitle alignment. The technique uses pose invariant features, extracted from detected faces and clustered to form groups of co-occurring characters. Results show that with 9 characters, 29% of the closest exemplars are correctly identified, increasing to 50% as additional exemplars are considered.

show abstract

“…In the absence of subtitles, Sankar et al [9] attempted to align script and movie using video cues, on-screen text (for silent movies) and speech-to-text transcript. When a wordlevel time accuracy is needed, a script-speech alignment using a speech recognition engine associated with a script-based language model such as described by Hong et al in [10] is best.…”

Section: Introductionmentioning

confidence: 99%

Scene reordering in movie script alignment

Lambert

Guégan

Zhou

2013

2013 11th International Workshop on Content-Based Multimedia Indexing (CBMI)

View full text Add to dashboard Cite

Screenplays are a very interesting source of information to describe and analyze a movie. Once aligned to the video timeline using time information extracted from the subtitles, they can be useful in many multimedia applications because they contain textual descriptions of scenes, names of the speaking characters, dialogues, etc.An alignment however is not always straightforward. The scripts freely available online often do not reflect the released version of the movie showing changes in dialogues, as well as scenes added, removed or reordered in post-production.In this paper we present two improvements to the algorithm traditionally used to align movie scripts with their videos. In particular we show that performance can be improved when taking into account the sequences that were reordered during the editing phase. We also present an automatic categorization of screenplays based on their similarity with the released movie.

show abstract

Subtitle-free Movie to Script Alignment

Cited by 31 publications

References 12 publications

Naming TV characters by watching and analyzing dialogs

Naming TV characters by watching and analyzing dialogs

Friendly Faces: Weakly Supervised Character Identification

Scene reordering in movie script alignment

Contact Info

Product

Resources

About