Multi-view Synchronization of Human Actions and Dynamic Scenes

Dexter, Emilie; Pérez, Patrick; Laptev, Ivan

doi:10.5244/c.23.122

Cited by 22 publications

(18 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The results shown in Table 1 Table 1: (C = Cricket, B = Baseball, F = Football) Synchronization Errors (in frames) using our proposed approach (WT), our proposed approach excluding the random sampling around the edge map (WT -EMS), the TE approach of [18], and the Selfsimilarity Matrices (SSM) approach of [8]. Our method performs significantly better for these classes of sports videos.…”

Section: Form Comparison Matrixmentioning

confidence: 97%

“…However, none of them is automatic and most of them do not consider multiple views under occlusions. The self-similarity descriptor (SSM) of [11] presents an unsupervised solution to view-invariant activity recognition and has been used for automatic synchronization of videos [8]. However, the results are promising only for videos with very cleanly extracted activities and in the absence of occlusions.…”

Section: View-invariant Activity Recognition -Problems and Related Workmentioning

confidence: 99%

“…Table 1 reports the mean synchronization frame error for our test set using the proposed approach (WT), the method of [8] using self-similarity matrices (SSMs), and the algorithm of [18] based on temporal signals from epipolar lines (TE). The TE method of [18] assumes the prior availability of the fundamental matrix, which is not available for the videos in our test set.…”

Section: Form Comparison Matrixmentioning

confidence: 99%

“…Dexter et al [8] proposed self-similarity matrices for describing motions under different viewing angles, and then used these descriptors for video synchronization. However, their work is robust only for simple actions without occlusions.…”

Section: Video Synchronization -Problems and Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Synchronization of user-generated videos through trajectory correspondence and a refinement procedure

Shankar

Lasenby

Kokaram

2013

Proceedings of the 10th European Conference on Visual Media Production

View full text Add to dashboard Cite

Temporal synchronization of multiple video recordings of the same dynamic event is a critical task in many computer vision applications e.g. novel view synthesis and 3D reconstruction. Typically this information is implied through the time-stamp information embedded in the video streams. User-generated videos shot using consumer grade equipment do not contain this information; hence, there is a need to temporally synchronize signals using the visual information itself. Previous work in this area has either assumed good quality data with relatively simple dynamic content or the availability of precise camera geometry.Our first contribution is a synchronization technique which tries to establish correspondence between feature trajectories across views in a novel way, and specifically targets the kind of complex content found in consumer generated sports recordings, without assuming precise knowledge of fundamental matrices or homographies. We evaluate performance using a number of real video recordings and show that our method is able to synchronize to within 1 sec, which is significantly better than previous approaches.Our second contribution is a robust and unsupervised viewinvariant activity recognition descriptor that exploits recurrence plot theory on spatial tiles. The descriptor is individually shown to better characterize the activities from different views under occlusions than state-of-the-art approaches. We combine this descriptor with our proposed synchronization method and show that it can further refine the synchronization index.

show abstract

Section: Form Comparison Matrixmentioning

confidence: 97%

Section: View-invariant Activity Recognition -Problems and Related Workmentioning

confidence: 99%

Section: Form Comparison Matrixmentioning

confidence: 99%

Section: Video Synchronization -Problems and Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Synchronization of user-generated videos through trajectory correspondence and a refinement procedure

Shankar

Lasenby

Kokaram

2013

Proceedings of the 10th European Conference on Visual Media Production

View full text Add to dashboard Cite

show abstract

“…The relative frame lag is measured either by comparing recorded motion present in the two video streams [41,42], or by actively inserting artificial optical cues into the field of vision of the cameras [43]. This avoids the need for explicit synchronisation and is touted as a means of reducing costs but there are a number of scenarios where the net complexity and cost is increased by the need of the additional post-processing step.…”

Section: Overview Of Synchronisation Techniquesmentioning

confidence: 99%

A high speed tri-vision system for automotive applications

Azzopardi

Grech

Leconte³

2010

Eur. Transp. Res. Rev.

View full text Add to dashboard Cite

Purpose Cameras are excellent ways of non-invasively monitoring the interior and exterior of vehicles. In particular, high speed stereovision and multivision systems are important for transport applications such as driver eye tracking or collision avoidance. This paper addresses the synchronisation problem which arises when multivision camera systems are used to capture the high speed motion common in such applications. Methods An experimental, high-speed tri-vision camera system intended for real-time driver eye-blink and saccade measurement was designed, developed, implemented and tested using prototype, ultra-high dynamic range, automotivegrade image sensors specifically developed by E2V (formerly Atmel) Grenoble SA as part of the European FP6 project -SENSATION (advanced sensor development for attention stress, vigilance and sleep/wakefulness monitoring). Results The developed system can sustain frame rates of 59.8 Hz at the full stereovision resolution of 1280×480 but this can reach 750 Hz when a 10 k pixel Region of Interest (ROI) is used, with a maximum global shutter speed of 1/48000 s and a shutter efficiency of 99.7%. The data can be reliably transmitted uncompressed over standard copper Camera-Link® cables over 5 metres. The synchronisation error between the left and right stereo images is less than 100 ps and this has been verified both electrically and optically. Synchronisation is automatically established at boot-up and maintained during resolution changes. A third camera in the set can be configured independently. The dynamic range of the 10bit sensors exceeds 123 dB with a spectral sensitivity extending well into the infra-red range. Conclusion The system was subjected to a comprehensive testing protocol, which confirms that the salient requirements for the driver monitoring application are adequately met and in some respects, exceeded. The synchronisation technique presented may also benefit several other automotive stereovision applications including near and farfield obstacle detection and collision avoidance, road condition monitoring and others.

show abstract