HiEve: A Large-Scale Benchmark for Human-Centric Video Analysis in Complex Events

Lin, Weiyao; Liu, Huabin; Liu, Shizhan; Li, Yuxi; Xiong, Hongkai; Qi, Guo-Jun; Sebe, Nicu

doi:10.1007/s11263-023-01842-6

Cited by 18 publications

(1 citation statement)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, tasks related to instructional videos are gaining increasing attention. Related datasets including COIN (Tang et al 2019), Diving (Li, Li, and Vasconcelos 2018), CSV (Qian et al 2022), EPIC-KITCHENS (Damen et al 2018), Assembly101 (Sener et al 2022), and HiEve (Lin et al 2023) have provided instructional videos in different scenarios. A prominent task for instructional videos is action segmentation, which aims to divide a video into successive steps.…”

Section: Related Workmentioning

confidence: 99%

Collaborative Weakly Supervised Video Correlation Learning for Procedure-Aware Instructional Video Analysis

He,

Liu,

et al. 2024

AAAI

View full text Add to dashboard Cite

Video Correlation Learning (VCL), which aims to analyze the relationships between videos, has been widely studied and applied in various general video tasks. However, applying VCL to instructional videos is still quite challenging due to their intrinsic procedural temporal structure. Specifically, procedural knowledge is critical for accurate correlation analyses on instructional videos. Nevertheless, current procedure-learning methods heavily rely on step-level annotations, which are costly and not scalable. To address this problem, we introduce a weakly supervised framework called Collaborative Procedure Alignment (CPA) for procedure-aware correlation learning on instructional videos. Our framework comprises two core modules: collaborative step mining and frame-to-step alignment. The collaborative step mining module enables simultaneous and consistent step segmentation for paired videos, leveraging the semantic and temporal similarity between frames. Based on the identified steps, the frame-to-step alignment module performs alignment between the frames and steps across videos. The alignment result serves as a measurement of the correlation distance between two videos. We instantiate our framework in two distinct instructional video tasks: sequence verification and action quality assessment. Extensive experiments validate the effectiveness of our approach in providing accurate and interpretable correlation analyses for instructional videos.

show abstract