Distilling Structured Knowledge into Embeddings for Explainable and Accurate Recommendation

Zhang, Yuan; Xu, Xiaoran; Zhou, Hu; Zhang, Yan

doi:10.1145/3336191.3371790

Cited by 37 publications

(12 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is achieved by designing a distillation loss function that exploits the teacher model's trained parameters to guide the student model's learning. Recently, a few attempts have been made to design knowledge distillation models in recommendation systems, following the collaborative filtering strategy and focusing on the ranking performance in the top-n recommendation task [16], [17], [34], [35]. However, these studies focus on the conventional recommendation task and are not suitable for conversational systems, where users progressively express their personalized preferences over conversational turns.…”

Section: Discussionmentioning

confidence: 99%

On Estimating the Training Cost of Conversational Recommendation Systems

Antaris,

Rafailidis,

Aliannejadi

2020

Preprint

View full text Add to dashboard Cite

Conversational recommendation systems have recently gain a lot of attention, as users can continuously interact with the system over multiple conversational turns. However, conversational recommendation systems are based on complex neural architectures, thus the training cost of such models is high. To shed light on the high computational training time of stateof-the art conversational models, we examine five representative strategies and demonstrate this issue. Furthermore, we discuss possible ways to cope with the high training cost following knowledge distillation strategies, where we detail the key challenges to reduce the online inference time of the high number of model parameters in conversational recommendation systems.

show abstract

Section: Discussionmentioning

confidence: 99%

On Estimating the Training Cost of Conversational Recommendation Systems

Antaris,

Rafailidis,

Aliannejadi

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Knowledge graphs can also help to explain a blank-box recommendation model. Zhang et al (2020) proposed a knowledge distillation approach to explain black-box models for recommendation. The authors proposed an end-to-end joint learning framework to combine the advantages of embedding-based recommendation models and path-based recommendation models.…”

Section: Knowledge Graph-based Explainable Recommendationmentioning

confidence: 99%

Explainable Recommendation: A Survey and New Perspectives

Zhang

Chen

2020

FNT in Information Retrieval

568

241

View full text Add to dashboard Cite

Explainable recommendation attempts to develop models that generate not only high-quality recommendations but also intuitive explanations. The explanations may either be post-hoc or directly come from an explainable model (also called interpretable or transparent model in some contexts). Explainable recommendation tries to address the problem of why: by providing explanations to users or system designers, it helps humans to understand why certain items are recommended by the algorithm, where the human can either be users or system designers. Explainable recommendation helps to improve the transparency, persuasiveness, effectiveness, trustworthiness, and satisfaction of recommendation systems. It also facilitates system designers for better system debugging. In recent years, a large number of explainable recommendation approaches -especially model-based methods -have been proposed and applied in real-world systems.In this survey, we provide a comprehensive review for the explainable recommendation research. We first highlight the position of explainable recommendation in recommender system research by categorizing recommendation problems into the 5W, i.e., what, when, who, where, and why. We then conduct a comprehensive survey of explainable recommendation on three perspectives: 1) We provide a chronological research timeline of explainable recommendation, including user study approaches in the early years and more recent model-based approaches. 2) We provide a two-dimensional taxonomy to classify existing explainable recommendation research: one dimension is the information source (or display style) of the explanations, and the other dimension is the algorithmic mechanism to generate explainable recommendations. 3) We summarize how explainable recommendation applies to different recommendation tasks, such as product recommendation, social recommendation, and POI recommendation.We also devote a section to discuss the explanation perspectives in broader IR and AI/ML research. We end the survey by discussing potential future directions to promote the explainable recommendation research area and beyond.Explainable recommendation refers to personalized recommendation algorithms that address the problem of why -they not only provide users or system designers with recommendation results, but also explanations to clarify why such items are recommended. In this way, it helps to improve the transparency, persuasiveness, effectiveness, trustworthiness, and user satisfaction of the recommendation systems. It also facilitates system designers to diagnose, debug, and refine the recommendation algorithm.To highlight the position of explainable recommendation in the recommender system research area, we classify personalized recommendation with a broad conceptual taxonomy. Specifically, personalized recommendation research can be classified into the 5W problemswhen, where, who, what, and why, corresponding to time-aware recommendation (when), location-based recommendation (where), social recommendation (who), application-aware re...

show abstract

“…Liu et al [6] leverage KD for debiasing in recommendation via uniform data. Zhang et al [31] study the mutual learning between path-based and embedding based recommendation models. Our work is partially inspired from these studies and serves as the first attempt to apply KD for ensembled CTR prediction.…”

Section: Knowledge Distillationmentioning

confidence: 99%

Ensembled CTR Prediction via Knowledge Distillation

Zhu

Liu

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Recently, deep learning-based models have been widely studied for click-through rate (CTR) prediction and lead to improved prediction accuracy in many industrial applications. However, current research focuses primarily on building complex network architectures to better capture sophisticated feature interactions and dynamic user behaviors. The increased model complexity may slow down online inference and hinder its adoption in real-time applications. Instead, our work targets at a new model training strategy based on knowledge distillation (KD). KD is a teacher-student learning framework to transfer knowledge learned from a teacher model to a student model. The KD strategy not only allows us to simplify the student model as a vanilla DNN model but also achieves significant accuracy improvements over the state-of-the-art teacher models. The benefits thus motivate us to further explore the use of a powerful ensemble of teachers for more accurate student model training. We also propose some novel techniques to facilitate ensembled CTR prediction, including teacher gating and early stopping by distillation loss. We conduct comprehensive experiments against 12 existing models and across three industrial datasets. Both offline and online A/B testing results show the effectiveness of our KD-based training strategy.

show abstract

Distilling Structured Knowledge into Embeddings for Explainable and Accurate Recommendation

Cited by 37 publications

References 43 publications

On Estimating the Training Cost of Conversational Recommendation Systems

On Estimating the Training Cost of Conversational Recommendation Systems

Explainable Recommendation: A Survey and New Perspectives

Ensembled CTR Prediction via Knowledge Distillation

Contact Info

Product

Resources

About