Self-Supervised Human Pose based Multi-Camera Video Synchronization

Liu, Yin; Han, Ruize; Feng, Wei; Wang, Song

doi:10.1145/3503161.3547766

Cited by 4 publications

(4 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare our method with all existing deep learningbased video synchronization methods, including SynNet (Wu et al 2019), SeSyn-Net (Yin et al 2022), and CNNSiamese (Boizard et al 2023). As the comparison can not be carried out directly, we will detail the comparison process.…”

Section: Competing Methodsmentioning

confidence: 99%

“…Therefore, subsequent research mainly adopted the strategy of synchronization based on pose features. As the dataset (Wu et al 2019) is not released to the public, the NTU RGB+D dataset (Shahroudy et al 2016) and CMU Panoptic Studio dataset (Joo et al 2015) was processed to obtain NTU-SYN Dataset and CMU-SYN Dataset (Yin et al 2022). Unfortunately, the time intervals in the dataset are still integer multiples of the frame interval.…”

Section: Video Temporal Synchronizationmentioning

confidence: 99%

“…However, these four datasets are not released to the public. The NTU-SYN and CMU-SYN datasets were processed (Yin et al 2022) and are now released to the public. We will subsequently introduce these two datasets along with our own and present the results of different methods applied to these three datasets.…”

Section: Experiments Datasetmentioning

confidence: 99%

“…However, labeling the time intervals between different views is challenging. One intuitive method uses synchronization devices to record synchronized videos and shifts an integral number of frames as the time interval (Wu et al 2019;Yin et al 2022;Boizard et al 2023). However, this method results in discrete labeled time intervals, with lengths that can only be multiples of a single frame.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Advancing Video Synchronization with Fractional Frame Analysis: Introducing a Novel Dataset and Model

Liu,

Ai,

Xing

et al. 2024

AAAI

View full text Add to dashboard Cite

Multiple views play a vital role in 3D pose estimation tasks. Ideally, multi-view 3D pose estimation tasks should directly utilize naturally collected videos for pose estimation. However, due to the constraints of video synchronization, existing methods often use expensive hardware devices to synchronize the initiation of cameras, which restricts most 3D pose collection scenarios to indoor settings. Some recent works learn deep neural networks to align desynchronized datasets derived from synchronized cameras and can only produce frame-level accuracy. For fractional frame video synchronization, this work proposes an Inter-Frame and Intra-Frame Desynchronized Dataset (IFID), which labels fractional time intervals between two video clips. IFID is the first dataset that annotates inter-frame and intra-frame intervals, with a total of 382,500 video clips annotated, making it the largest dataset to date. We also develop a novel model based on the Transformer architecture, named InSynFormer, for synchronizing inter-frame and intra-frame. Extensive experimental evaluations demonstrate its promising performance. The dataset and source code of the model are available at https://github.com/yuxuan-cser/InSynFormer.

show abstract

Section: Competing Methodsmentioning

confidence: 99%