DOI: 10.31390/gradschool_dissertations.5426
|View full text |Cite
|
Sign up to set email alerts
|

Exploring Knowledge Transfer and Distillation in Deep Learning

Abstract: Nighttime unmanned aerial vehicle (UAV) tracking has been facilitated with indispensable plug-and-play low-light enhancers. However, the introduction of low-light enhancers increases the extra computational burden for the UAV, significantly hindering the development of real-time UAV applications. Meanwhile, these state-of-the-art (SOTA) enhancers lack tight coupling with the advanced daytime UAV tracking approach. To solve the above issues, this work proposes a novel mutuallearning knowledge distillation frame… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1

Citation Types

0
1
0

Publication Types

Select...
1

Relationship

0
1

Authors

Journals

citations
Cited by 1 publication
(1 citation statement)
references
References 32 publications
0
1
0
Order By: Relevance
“…The fundamental concept underlying Knowledge Distillation (KD) involves utilizing acquired knowledge (e.g., logits [36], feature values [35,37,38,40,78], and sample relations [51,66]) from a high-capacity teacher to guide the training of a student model. The training dataset (X, Y ) comprises training samples X = x i n i=1 and their corresponding labels Y = y i n i=1 .…”
Section: General Formulation Of Knowledge Distillationmentioning
confidence: 99%
“…The fundamental concept underlying Knowledge Distillation (KD) involves utilizing acquired knowledge (e.g., logits [36], feature values [35,37,38,40,78], and sample relations [51,66]) from a high-capacity teacher to guide the training of a student model. The training dataset (X, Y ) comprises training samples X = x i n i=1 and their corresponding labels Y = y i n i=1 .…”
Section: General Formulation Of Knowledge Distillationmentioning
confidence: 99%