Exploiting Script-Subtitles Alignment to Scene Boundary Dectection in Movie

Abstract

A movie, a typical multimedia data, includes several textual information such as subtitles and annotation texts as well as audio-visual information. In general, a story of a movie is progressed by dialogues of characters and the dialogues are produced based on a pre-written script that contains detailed descriptions of all the information related to the movie. The script consists of scene headings, character names, dialogues, actions, and so on. Rich information related to the movie can be extracted easily and accurately from each component of the script. However, since the script does not contain time information, additional processes are required to synchronize the script with the movie. The subtitle is usually made from the script's dialogue and has time information to display text on screen when the characters speak. Therefore, synchronization between the script and the movie can be accomplished through an alignment process between the script's dialogue and its' subtitle. In this paper, we propose a unique method of scene boundary detection exploited scriptsubtitles alignment. By synchronizing the script to movie, we identify scene boundaries with semantic level. In addition, we build a thesaurus to enhance alignment performance. Experimental results with real movies show that the proposed method can successfully, not only align scripts with those subtitles, but also detect scene boundaries.

Scene reordering in movie script alignment

“…Park et al [12] complement the naïve use of dynamic programming with a thesaurus and achieve significant improvements. Our work does not make use of any external knowledge and both approaches could benefit from each other.…”

“…The benefits of parsing the screenplay before the alignment are two-fold: first it separates the dialogues from the rest of the text, thus avoiding misalignment of the subtitles to text description as could be the case in [12], and secondly it allows for easy access to the different elements (each category is given a specific tag in the final XML file).…”

“…In [12] Park et al suggest the use of a ratio between the script and subtitle sentences as a threshold, rejecting the alignments scoring less than 50%. Laptev et al in [5] compute a similar ratio.…”

“…We can however compare our work with results from Park et al [12] who also included the movie "Notting Hill" (NH) in their evaluation: Our respective baselines ("Naïve DP" and NW 1 ) are implementations of comparable alignment algorithms. The difference in accuracy is due to the additional parsing step that separates the DE from the other elements of the script in our system.…”

Scene Boundary Detection from Movie Dialogue: A Genetic Algorithm Approach

“…Scene boundary detection has also been attempted with the aid of scripts and subtitles. The scriptsubtitles alignment mechanism has been used in [7][8][9] for the purpose. It may be mentioned in this context that a script, unlike its subtitle counterpart does not convey any information about the time period in the movie.…”