ConCor+: Robust and confident video synchronization using consensus-based Cross-correlation

Abstract

Consensus-based Cross-correlation (ConCor) is a recently presented algorithm for robust synchronization of noisy and corrupted signals. ConCor has a number of interdependent parameters that need to be set correctly to guarantee good performance. In this paper we analyse the effects of the individual parameters on ConCor's behaviour and performance. As a second contribution, we show that a parameter sweep with subsequent majority voting can be used to boost ConCor's performance and produce a trustworthy confidence measure. As a final contribution we show how the proposed extension also allows performing multi-modal (joint audio-video) synchronization of casual multi-perspective video recordings enabling superior matching performance.

Video Synchronization With Bit-Rate Signals and Correntropy Function

“…A false estimation that happens as a result of an uncorrelated segment analysis is called an outlier. In another work found in the literature [ 25 ], the RANSAC is used to deal with the presence of outliers. The RANSAC algorithm estimates the parameters of a model in a two-step loop: the first step estimates the model parameters from a minimal sample set that is randomly selected; the second step checks which elements of the entire dataset are consistent with the model instantiated with the parameters estimated in the first step.…”

“…Recent studies [ 24 , 25 ] present the properties and usage of a signal extracted from stream instantaneous bit-rate, which is the VBR, to retrieve timing information of recorded data from multiple cameras capturing the same scene. Al-Nuaimi et al [ 25 ] developed an algorithm that is based on a measure called “consensus-based correlation”, or ConCor+, which uses a cross-correlation measure of pairwise VBR signals from multiple streams in conjunction with the random sample consensus algorithm (RANSAC) for timing classification.…”

“…The studies concerning the use of instantaneous bit-rate as a time descriptor [ 24 , 25 ] are the fundamental inspiration for devising our online synchronization method. The ConCor+ algorithm presented by Al-Nuaimi et al [ 25 ] is our closest scientific reference with comparable results of those discussed in Section 4 .…”

Towards the design of an intuitive multi-view video navigation interface based on spatial information

“…The server side of our system has the task of receiving the related video recordings and the spatial information from the client application, applying the synchronization algorithm described in Al-Nuaimi et al's work [1] which is an extension of the algorithm presented by Schweiger et al [13] and storing all resulting information in a database. The content can then be accessed remotely via a Representational State Transfer (REST) interface.…”

“…Nowadays, video portals such as YouTube 1 display these perspectives as being related, 1 YouTube video portal -http://www.youtube.com depending on their associated metadata. However, the playout is done individually and no use is made of the spatial or temporal relationships of the videos to provide an interactive viewing experience with the possibility to switch perspectives on demand.…”