Movie synchronization by audio landmark matching

Abstract

International audienceThis paper addresses movie synchronization, i.e. synchronizing multiple versions of the same movie, with an objective of automatically transferring metadata available on a reference version to other ones. We first exploit audio tracks associated with two different versions and adapt an existing audio fingerprinting technique to find all temporal matching positions between them. We then propose additional steps to refine the match and eliminate outliers. The proposed approach can efficiently handle situations where temporal scene edits occur like scene addition, removal, and even the challenging scene reordering case. Experimental results over synthetic editorial data show the effectiveness of the proposed approach with respect to the state-of-the-art dynamic time warping (DTW) based solution

Audio Fingerprinting for Multi-Device Self-Localization

“…A hop size as small as (equals s kHz) is used in the proposed algorithm so that an improved temporal analysis resolution is achieved and the TDOAs of different sources can be distinguished from each other in the histogram (14) as different peaks. This is in contrast to the choice in traditional audio fingerprinting techniques which have been applied to video synchronization [49] [50] or music information retrieval [53]. In these applications, the hop size is usually chosen to be a value within the range (equals s kHz), which is already enough for coarsely synchronizing audio channels but far below the requirement for TDOA and distance estimation.…”

“…Landmark-based audio fingerprinting is generally used for coarsely synchronizing audio recordings [12], [49], [50]. However, the extracted landmark features contain some valuable information about the TDOA information of the sound sources.…”

Fine landmark-based synchronization of ad-hoc microphone arrays

“…Traditional microphone-array techniques, such as beamforming and sound source localization, which rely on the knowledge of microphone positions and assume samplesynchronized audio channels, cannot be applied directly [2,3]. The synchronization problem between multiple audio channels has been addressed using generalized cross-correlation [2,4,5] and audio fingerprinting [5][6][7][8].…”

“…able for audio channels that are already coarsely synchronized [4,5]. Another synchronization approach is based on audio fingerprinting, which has been originally applied to music information retrieval [6], and clustering and synchronizing multi-camera videos [7,8]. By matching the audio fingerprints extracted from the sound track, the audio channels can be synchronized.…”

“…PRH extracts more detailed features including all the energy changes in consecutive bands and frames, and is more robust than onset to ambient noise and audio compression distortion [12]. Audio landmark operates in the time-frequency domain, and the extracted features consist of a set of timefrequency pairs [6][7][8]. With low computational cost and robustness against noise, audio fingerprinting techniques have been widely used in movie and video synchronization [13].…”