Introducing LensKit-Auto, an Experimental Automated Recommender System (AutoRecSys) Toolkit

Vente, Tobias; Ekstrand, Michael; Beel, Joeran

doi:10.1145/3604915.3610656

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Other2

Book1

Relationship

Self Cite0

Independent3

Authors

Journals

Cited by 3 publications

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Revealing the Hidden Impact of Top-N Metrics on Optimization in Recommender Systems

Wegmeth,

Vente,

Purucker

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Revealing the Hidden Impact of Top-N Metrics on Optimization in Recommender Systems

Wegmeth,

Vente,

Purucker

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

From Clicks to Carbon: The Environmental Toll of Recommender Systems

Vente,

Wegmeth,

Said

et al. 2024

18th ACM Conference on Recommender Systems

View full text Add to dashboard Cite

Recommender Systems Algorithm Selection for Ranking Prediction on Implicit Feedback Datasets

Wegmeth,

Vente,

Beel

2024

18th ACM Conference on Recommender Systems

View full text Add to dashboard Cite

The recommender systems algorithm selection problem for ranking prediction on implicit feedback datasets is under-explored. Traditional approaches in recommender systems algorithm selection focus predominantly on rating prediction on explicit feedback datasets, leaving a research gap for ranking prediction on implicit feedback datasets. Algorithm selection is a critical challenge for nearly every practitioner in recommender systems. In this work, we take the first steps toward addressing this research gap.We evaluate the NDCG@10 of 24 recommender systems algorithms, each with two hyperparameter configurations, on 72 recommender systems datasets. We train four optimized machine-learning meta-models and one automated machine-learning meta-model with three different settings on the resulting meta-dataset.Our results show that the predictions of all tested meta-models exhibit a median Spearman correlation ranging from 0.857 to 0.918 with the ground truth. We show that the median Spearman correlation between meta-model predictions and the ground truth increases by an average of 0.124 when the meta-model is optimized to predict the ranking of algorithms instead of their performance. Furthermore, in terms of predicting the best algorithm for an unknown dataset, we demonstrate that the best optimized traditional meta-model, e.g., XGBoost, achieves a recall of 48.6%, outperforming the best tested automated machine learning meta-model, e.g., AutoGluon, which achieves a recall of 47.2%.

show abstract

Introducing LensKit-Auto, an Experimental Automated Recommender System (AutoRecSys) Toolkit

Cited by 3 publications

References 46 publications

Revealing the Hidden Impact of Top-N Metrics on Optimization in Recommender Systems

Revealing the Hidden Impact of Top-N Metrics on Optimization in Recommender Systems

From Clicks to Carbon: The Environmental Toll of Recommender Systems

Recommender Systems Algorithm Selection for Ranking Prediction on Implicit Feedback Datasets

Contact Info

Product

Resources

About