Knowledge distillation for multi-depth-model-fusion recommendation algorithm

Abstract

Recommendation algorithms save a lot of valuable time for people to get the information they are interested in. However, the feature calculation and extraction process of each machine learning or deep learning recommendation algorithm are different, so how to obtain various features with different dimensions, i.e., how to integrate the advantages of each model and improve the model inference efficiency, becomes the focus of this paper. In this paper, a better deep learning model is obtained by integrating several cutting-edge deep learning models. Meanwhile, to make the integrated learning model converge better and faster, the parameters of the integrated module are initialized, constraints are imposed, and a new activation function is designed for better integration of the sub-models. Finally, the integrated large model is distilled for knowledge distillation, which greatly reduces the number of model parameters and improves the model inference efficiency.