Adaptive multi-teacher softened relational knowledge distillation framework for payload mismatch in image steganalysis

Yu, Lifang; Li, Yunwei; Weng, Shaowei; Tian, Huawei; Liu, Jing

doi:10.1016/j.jvcir.2023.103900

Cited by 3 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The student model gains the teacher model's knowledge through distillation training, with a potential slight performance loss traded for transferring knowledge from complex teacher models to a simpler student model. Currently, most instances of knowledge distillation focus solely on a single teacher model, overlooking the possibility that a single student model can be supervised by multiple teacher models [20,21]. Alternatively, recognizing the equal significance of multiple teacher models renders it impractical to acquire more valid knowledge based on inherent distinctions in these models.…”

Section: Knowledge Distillationmentioning

confidence: 99%

Research and Application of Edge Computing and Deep Learning in a Recommender System

Hao,

Shan,

Zhang

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Recommendation systems play a pivotal role in improving product competitiveness. Traditional recommendation models predominantly use centralized feature processing to operate, leading to issues such as excessive resource consumption and low real-time recommendation concurrency. This paper introduces a recommendation model founded on deep learning, incorporating edge computing and knowledge distillation to address these challenges. Recognizing the intricate relationship between the accuracy of deep learning algorithms and their complexity, our model employs knowledge distillation to compress deep learning. Teacher–student models were initially chosen and constructed in the cloud, focusing on developing structurally complex teacher models that incorporate passenger and production characteristics. The knowledge acquired from these models was then transferred to a student model, characterized by weaker learning capabilities and a simpler structure, facilitating the compression and acceleration of an intelligent ranking model. Following this, the student model underwent segmentation, and certain computational tasks were shifted to end devices, aligning with edge computing principles. This collaborative approach between the cloud and end devices enabled the realization of an intelligent ranking for product listings. Finally, a random selection of the passengers’ travel records from the last five years was taken to test the accuracy and performance of the proposed model, as well as to validate the intelligent ranking of the remaining tickets. The results indicate that, on the one hand, an intelligent recommendation system based on knowledge distillation and edge computing successfully achieved the concurrency and timeliness of the existing remaining ticket queries. Simultaneously, it guaranteed a certain level of accuracy, and reduced computing resource and traffic load on the cloud, showcasing its potential applicability in highly concurrent recommendation service scenarios.

show abstract

Section: Knowledge Distillationmentioning

confidence: 99%