Bootstrapping User and Item Representations for One-Class Collaborative Filtering

Lee, Dongha; Kang, SeongKu; Ju, Hyunjun; Park, Chanyoung; Yu, Hwanjo

doi:10.1145/3404835.3462935

Cited by 75 publications

(38 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When the batch size is too small, the topology cannot include the overall relational knowledge in the representation space, leading to limited performance. For CiteULike, we observe that HTD achieves the stable performance around 2 8 -2 11 . In this work, we set the batch size to 2 10 .…”

Section: Hyperparameter Analysismentioning

confidence: 87%

See 1 more Smart Citation

Topology Distillation for Recommender System

Kang

Hwang

Kweon

et al. 2021

Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery &Amp; Data Mining

Self Cite

View full text Add to dashboard Cite

Recommender Systems (RS) have employed knowledge distillation which is a model compression technique training a compact student model with the knowledge transferred from a pre-trained large teacher model. Recent work has shown that transferring knowledge from the teacher's intermediate layer significantly improves the recommendation quality of the student. However, they transfer the knowledge of individual representation point-wise and thus have a limitation in that primary information of RS lies in the relations in the representation space. This paper proposes a new topology distillation approach that guides the student by transferring the topological structure built upon the relations in the teacher space. We first observe that simply making the student learn the whole topological structure is not always effective and even degrades the student's performance. We demonstrate that because the capacity of the student is highly limited compared to that of the teacher, learning the whole topological structure is daunting for the student. To address this issue, we propose a novel method named Hierarchical Topology Distillation (HTD) which distills the topology hierarchically to cope with the large capacity gap. Our extensive experiments on real-world datasets show that the proposed method significantly outperforms the state-of-the-art competitors. We also provide in-depth analyses to ascertain the benefit of distilling the topology for RS. CCS CONCEPTS• Information systems → Learning to rank; Collaborative filtering; Retrieval efficiency.

show abstract

Section: Hyperparameter Analysismentioning

confidence: 87%

“…We use two public real-world datasets: CiteULike and Foursquare. We remove users having fewer than 5 (CiteULike) and 20 interactions (FourSquare) and remove items having fewer than 10 interactions (FourSquare) as done in [8,11]. Table 5 summarizes the statistics of the datasets.…”

Section: A Appendix A1 Pseudocode Of the Proposed Methodsmentioning

confidence: 99%

Topology Distillation for Recommender System

Kang

Hwang

Kweon

et al. 2021

Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery &Amp; Data Mining

Self Cite

View full text Add to dashboard Cite

show abstract

“…We use three real-world datasets: CiteULike [53], Ciao [50], and Foursquare [38]. These datasets are publicly available and widely used in recent studies [16,20,28]. We follow the preprocessing of [28].…”

Section: A Appendix A1 Experiments Setupmentioning

confidence: 99%

“…These datasets are publicly available and widely used in recent studies [16,20,28]. We follow the preprocessing of [28]. Table 6 summarizes the statistics of the datasets.…”

Section: A Appendix A1 Experiments Setupmentioning

confidence: 99%

“…One-Class Collaborative Filtering (OCCF) aims to discover users' preferences and recommend the items that they might like in the future, given a set of observed user-item interactions (e.g., clicks or purchases) [28,43]. To effectively learn users' preferences from such implicit feedback, many learning objectives (or objective functions) have been researched based on a variety of underlying probabilistic models.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Consensus Learning from Heterogeneous Objectives for One-Class Collaborative Filtering

Kang¹,

Lee²,

Kweon³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Over the past decades, for One-Class Collaborative Filtering (OCCF), many learning objectives have been researched based on a variety of underlying probabilistic models. From our analysis, we observe that models trained with different OCCF objectives capture distinct aspects of user-item relationships, which in turn produces complementary recommendations. This paper proposes a novel OCCF framework, named as ConCF, that exploits the complementarity from heterogeneous objectives throughout the training process, generating a more generalizable model. ConCF constructs a multibranch variant of a given target model by adding auxiliary heads, each of which is trained with heterogeneous objectives. Then, it generates consensus by consolidating the various views from the heads, and guides the heads based on the consensus. The heads are collaboratively evolved based on their complementarity throughout the training, which again results in generating more accurate consensus iteratively. After training, we convert the multi-branch architecture back to the original target model by removing the auxiliary heads, thus there is no extra inference cost for the deployment. Our extensive experiments on real-world datasets demonstrate that ConCF significantly improves the generalization of the model by exploiting the complementarity from heterogeneous objectives. CCS CONCEPTS• Information systems → Collaborative filtering; • Computing methodologies → Learning from implicit feedback.

show abstract