GAZED Gaze-guided Cinematic Editing of Wide-Angle Monocular Video Recordings

Moorthy, K L Bhanu; Kumar, Moneish; Subramaniam, Ramanathan; Gandhi, Vineet

doi:10.1145/3313831.3376544

Cited by 18 publications

(1 citation statement)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Video saliency prediction aims at estimating patterns of human attention during free-viewing of dynamic scenes, to emulate the capabilities of the human visual system of quickly analyzing and interpreting the surrounding environment. Due to its several practical applications [7,5,28,33,9,40,12,27], it is an active area of research in computer vision. However, the solution to this problem is not trivial, for several reasons.…”

Section: Introductionmentioning

confidence: 99%

TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

Palazzo

Salanitri

et al. 2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Video saliency prediction has recently attracted attention of the research community, as it is an upstream task for several practical applications. However, current solutions are particularly computationally demanding, especially due to the wide usage of spatio-temporal 3D convolutions. We observe that, while different model architectures achieve similar performance on benchmarks, visual variations between predicted saliency maps are still significant. Inspired by this intuition, we propose a lightweight model that employs multiple simple heterogeneous decoders and adopts several practical approaches to improve accuracy while keeping computational costs low, such as hierarchical multi-map knowledge distillation, multi-output saliency prediction, unlabeled auxiliary datasets and channel reduction with teacher assistant supervision. Our approach achieves saliency prediction accuracy on par or better than state-of-the-art methods on DFH1K, UCF-Sports and Hol-lywood2 benchmarks, while enhancing significantly the efficiency of the model.

show abstract

Section: Introductionmentioning

confidence: 99%

TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

Palazzo

Salanitri

et al. 2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

show abstract

Attention-based automatic editing of virtual lectures for reduced production labor and effective learning experience

Hwang,

Lee

2024

International Journal of Human-Computer Studies

View full text Add to dashboard Cite

Salient Face Prediction without Bells and Whistles

Jyoti,

Girmaji,

Agrawal

et al. 2022

2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)

View full text Add to dashboard Cite

GAZED Gaze-guided Cinematic Editing of Wide-Angle Monocular Video Recordings

Abstract: methods is demonstrated via a psychophysical study involving 12 users and twelve performance videos.

Cited by 18 publications

References 24 publications

TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

Attention-based automatic editing of virtual lectures for reduced production labor and effective learning experience

Salient Face Prediction without Bells and Whistles

Contact Info

Product

Resources

About

GAZED Gaze-guided Cinematic Editing of Wide-Angle Monocular Video Recordings

Abstract: methods is demonstrated via a psychophysical study involving 12 users and twelve performance videos.

Cited by 18 publications

References 24 publications

TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

Attention-based automatic editing of virtual lectures for reduced production labor and effective learning experience

Salient Face Prediction without Bells and Whistles

Contact Info

Product

Resources

About

GAZED Gaze-guided Cinematic Editing of Wide-Angle Monocular Video Recordings