Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency

Li, Jia; Fu, Kui; Zhao, Shengwei; Ge, Shiming

doi:10.1109/tip.2019.2946102

Cited by 29 publications

(6 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several knowledge distillation approaches have been recently proposed for video saliency prediction. SKD-DVA [20] proposes spatio-temporal knowledge distillation with two teachers and two students, with each pair focusing on either spatial or temporal transfer. SV2T-SS [41] distills corresponding features of teacher and student (implemented as encoderdecoder networks), based on first-and second-order feature statistics transfer.…”

Section: Knowledge Distillation For Visual Saliency Predictionmentioning

confidence: 99%

TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

Palazzo

Salanitri

et al. 2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Video saliency prediction has recently attracted attention of the research community, as it is an upstream task for several practical applications. However, current solutions are particularly computationally demanding, especially due to the wide usage of spatio-temporal 3D convolutions. We observe that, while different model architectures achieve similar performance on benchmarks, visual variations between predicted saliency maps are still significant. Inspired by this intuition, we propose a lightweight model that employs multiple simple heterogeneous decoders and adopts several practical approaches to improve accuracy while keeping computational costs low, such as hierarchical multi-map knowledge distillation, multi-output saliency prediction, unlabeled auxiliary datasets and channel reduction with teacher assistant supervision. Our approach achieves saliency prediction accuracy on par or better than state-of-the-art methods on DFH1K, UCF-Sports and Hol-lywood2 benchmarks, while enhancing significantly the efficiency of the model.

show abstract

Section: Knowledge Distillation For Visual Saliency Predictionmentioning

confidence: 99%

TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

Palazzo

Salanitri

et al. 2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

show abstract

“…Several works have already used KD to obtain lightweight models suitable for UAVs. For example, Li et al [16] have applied this technique for video saliency estimation, while Liu et al [17], Yu [18], Ding et al [19], and Luo et al [20] used it for object detection, object recognition, action recognition, and UAV delivery, respectively. However, to our knowledge, no work has investigated knowledge distillation to produce efficient and accurate models tailored for UAVs in the context of weed mapping.…”

Section: B Knowledge Distillationmentioning

confidence: 99%

Applying Knowledge Distillation to Improve Weed Mapping With Drones

Castellano,

Marinis,

Vessio

2023

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

In precision agriculture, non-invasive remote sensing using UAVs can be employed to observe crops in visible and nonvisible spectra. This paper investigates the effectiveness of stateof-the-art knowledge distillation techniques for mapping weeds with drones, an essential component of precision agriculture that employs remote sensing to monitor crops and weeds. The study introduces a lightweight Vision Transformer-based model that achieves optimal weed mapping capabilities while maintaining minimal computation time. The research shows that the student model effectively learns from the teacher model using the WeedMap dataset, achieving accurate results suitable for mobile platforms such as drones, with only 0.5 GMacs compared to 42.5 GMacs of the teacher model. The trained models obtained an F1 score of 0.863 and 0.631 on two data subsets, with a performance improvement of 2 and 7 points, respectively, over the undistilled model. The study results suggest that developing efficient computer vision algorithms on drones can significantly improve agricultural management practices, leading to greater profitability and environmental sustainability.

show abstract

“…Guo et al [22] improved the student network to produce more confident predictions with the help of the teacher network for robust student network learning. Li et al [23] proposed a dynamic saliency estimation approach for aerial videos via spatiotemporal knowledge distillation, which could effectively remove the inter-model redundancy. Zhang et al [24] learned to distill future knowledge from a backward neural language model (teacher) to future-aware vectors (student) during the training phase, which were incorporated into the attention layer to provide full-range context information.…”

Section: Related Workmentioning

confidence: 99%

Variational Bayesian Group-Level Sparsification for Knowledge Distillation

Ming

Jiang³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Deep neural networks are capable of learning powerful representation, but often limited by heavy network architectures and high computational cost. Knowledge distillation (KD) is one of the effective ways to perform model compression and inference acceleration. But the final student models remain parameter redundancy. To tackle these issues, we propose a novel approach, called Variational Bayesian Group-level Sparsification for Knowledge Distillation (VBGS-KD), to distill a large teacher network into a small and sparse student network while preserving accuracy. We impose the sparsity-inducing prior on the groups of parameters in the student model, and introduce the variational Bayesian approximation to learn structural sparseness, which can effectively prune most part of weights. The prune threshold is learned during training without extra fine-tuning. The proposed method can learn the robust student networks that have achieved satisifying accuracy and compact sizes compared with the state-of-the-arts methods. We have validated our method on the MNIST, and CIFAR-10 datasets, observing 90.3% sparsity with 0.19% accuracy boosting in MNIST. Extensive experiments on the CIFAR-10 dataset demonstrate the efficiency of the proposed approach. INDEX TERMS Knowledge distillation, group sparsity, sparsity-inducing prior, variational Bayesian approximation.

show abstract

Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency

Cited by 29 publications

References 57 publications

TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation

Applying Knowledge Distillation to Improve Weed Mapping With Drones

Variational Bayesian Group-Level Sparsification for Knowledge Distillation

Contact Info

Product

Resources

About