Causality-Aware Neighborhood Methods for Recommender Systems

Sato, Masahiro; Singh, Janmajay; Takemori, Sho; Zhang, Qian

doi:10.1007/978-3-030-72113-8_40

Cited by 7 publications

(7 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DLCE [27] is an unbiased learning method for the causal effect that uses an IPS-based unbiased learning objective. There are also neighborhood methods for causal effects [28] that are based on a matching estimator in causal inference. These prior works on causal effects evaluated methods offline and did not discuss protocols for online evaluation.…”

Section: Recommendation Methods For the Causal Effectmentioning

confidence: 99%

“…Because we observe 𝑌 𝑢𝑖 = 𝑌 T 𝑢𝑖 if 𝑍 𝑢𝑖 = 1 and 𝑌 𝑢𝑖 = 𝑌 C 𝑢𝑖 if 𝑍 𝑢𝑖 = 0, both potential outcomes are necessary to simulate user outcomes under various ranking models and online evaluation methods. Following the procedure described in [28], we generated two datasets: one is based on the Dunnhumby dataset, 7 and the other is based on the MovieLens-1M (ML-1M) dataset [7]. 8 The detail and rationale of ML one are described in Section 5.1 of [28] and that of DH one are described in 5.1.1 of [27].…”

Section: Methodsmentioning

confidence: 99%

“…Following the procedure described in [28], we generated two datasets: one is based on the Dunnhumby dataset, 7 and the other is based on the MovieLens-1M (ML-1M) dataset [7]. 8 The detail and rationale of ML one are described in Section 5.1 of [28] and that of DH one are described in 5.1.1 of [27]. Each dataset is comprised of independently generated training and testing data.…”

Section: Methodsmentioning

confidence: 99%

“…Each dataset is comprised of independently generated training and testing data. The testing data were used to simulate online evaluation, and the training data were used to train the following models: 9 the causality-aware user-based neighborhood methods (CUBN) with outcome similarity (-O) and treatment similarity (-T) [28], the uplift-based pointwise and pairwise learning methods (ULRMF and ULBPR) [25], the Bayesian personalized ranking method (BPR) [20], and the user-based neighborhood method (UBN) [17]. We compared two models among CUBN-T, ULRMF, BPR on the Dunnhumby data and two models among CUBN-O, ULBPR, UBN on the ML-1M data.…”

Section: Methodsmentioning

confidence: 99%

“…While most recommendation methods aim for accurate predictions of user behaviors, there may be a discrepancy between the accuracy and the causal effect of recommendations [25]. Several recent works have thus proposed recommendation methods to rank items by the causal effect of recommendations [1,24,25,27,28].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Online Evaluation Methods for the Causal Effect of Recommendations

Sato¹

2021

Fifteenth ACM Conference on Recommender Systems

Self Cite

View full text Add to dashboard Cite

Evaluating the causal effect of recommendations is an important objective because the causal effect on user interactions can directly leads to an increase in sales and user engagement. To select an optimal recommendation model, it is common to conduct A/B testing to compare model performance. However, A/B testing of causal effects requires a large number of users, making such experiments costly and risky. We therefore propose the first interleaving methods that can efficiently compare recommendation models in terms of causal effects. In contrast to conventional interleaving methods, we measure the outcomes of both items on an interleaved list and items not on the interleaved list, since the causal effect is the difference between outcomes with and without recommendations. To ensure that the evaluations are unbiased, we either select items with equal probability or weight the outcomes using inverse propensity scores. We then verify the unbiasedness and efficiency of online evaluation methods through simulated online experiments. The results indicate that our proposed methods are unbiased and that they have superior efficiency to A/B testing.

show abstract

Section: Recommendation Methods For the Causal Effectmentioning

confidence: 99%