ContentWise Impressions

Maurera, Fernando Benjamín Pérez; Dacrema, Maurizio Ferrari; Saule, Lorenzo; Scriminaci, Mario; Cremonesi, Paolo

doi:10.1145/3340531.3412774

Cited by 16 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The same phenomenon can be observed for recommender systems with a carousel interface. Figure 2 shows the number of interactions for items displayed in the carousel interface of a video-on-demand service, we can again see how the user interactions are concentrated in the top-left corner [15]. To the best of our knowledge, no ranking metric that takes this behavior into account exists.…”

Section: Two-dimensional Ndcgmentioning

confidence: 99%

“…Another advantage of ContentWise Impressions is that it is subsampled in a way to be easily usable for research purposes without requiring significant computation resources. While researchers can indeed preprocess and subsample bigger datasets, if needed, this may result in different articles relying on different subsampling, Figure 2: Heatmap of the number of interactions per position on the screen, taken from the dataset presented in [15].…”

Section: Comparison With Other Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

A Methodology for the Offline Evaluation of Recommender Systems in a User Interface with Multiple Carousels

Felicioni

Dacrema

Cremonesi

2021

Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization

Self Cite

View full text Add to dashboard Cite

Many video-on-demand and music streaming services provide the user with a page consisting of several recommendation lists, i.e., widgets or swipeable carousels, each built with a specific criterion (e.g., most recent, TV series, etc.). Finding efficient strategies to select which carousels to display is an active research topic of great industrial interest. In this setting, the overall quality of the recommendations of a new algorithm cannot be assessed by measuring solely its individual recommendation quality. Rather, it should be evaluated in a context where other recommendation lists are already available, to account for how they complement each other. This is not considered by traditional offline evaluation protocols. Hence, we propose an offline evaluation protocol for a carousel setting in which the recommendation quality of a model is measured by how much it improves upon that of an already available set of carousels. We report experiments on publicly available datasets on the movie domain and notice that under a carousel setting the ranking of the algorithms change. In particular, when a SLIM carousel is available, matrix factorization models tend to be preferred, while item-based models are penalized. We also propose to extend ranking metrics to the two-dimensional carousel layout in order to account for a known position bias, i.e., users will not explore the lists sequentially, but rather concentrate on the top-left corner of the screen.

show abstract

Section: Two-dimensional Ndcgmentioning

confidence: 99%

Section: Comparison With Other Datasetsmentioning

confidence: 99%

A Methodology for the Offline Evaluation of Recommender Systems in a User Interface with Multiple Carousels

Felicioni

Dacrema

Cremonesi

2021

Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization

Self Cite

View full text Add to dashboard Cite

show abstract

Offline recommender system evaluation: Challenges and new directions

Castells

Moffat

2022

AI Magazine

View full text Add to dashboard Cite

Offline evaluation is an essential complement to online experiments in the selection, improvement, tuning, and deployment of recommender systems. Offline methodologies for recommender system evaluation evolved from experimental practice in Machine Learning (ML) and Information Retrieval (IR). However, evaluating recommendations involves particularities that pose challenges to the assumptions upon which the ML and IR methodologies were developed. We recap and reflect on the development and current status of recommender system evaluation, providing an updated perspective. With a focus on offline evaluation, we review the adaptation of IR principles, procedures and metrics, and the implications of those techniques when applied to recommender systems. At the same time, we identify the singularities of recommendation that require different responses, or involve specific new needs. In addition, we provide an overview of important choices in the configuration of experiments that require particular care and understanding; discuss broader perspectives of evaluation such as recommendation value beyond accuracy; and survey open challenges such as experimental biases, and the cyclic dimension of recommendation.

show abstract

From Data Analysis to Intent-Based Recommendation: An Industrial Case Study in the Video Domain

et al. 2022

View full text Add to dashboard Cite

This work presents a comprehensive study, from an industrial perspective, of the process between the collection of raw data, and the generation of next-item recommendation, in the domain of Video-on-Demand (VoD). Most research papers focus their efforts on analyzing recommender systems on already-processed datasets, but they do not face the same challenges that occur naturally in industry, e.g., processing raw interactions logs to create datasets for testing. This paper describes the whole process between data collection and recommendation, including cleaning, processing, feature engineering, session inferring, and all the challenges that a dataset provided by an industrial player in the domain posed. Then, a comparison on the new dataset of several intent-based recommendation techniques in the nextitem recommendation task follows, studying the impact of different factors like the session length, and the number of previous sessions available for a user. The results show that taking advantage of the sequential data available in the dataset benefits recommendation quality, since deep learning algorithms for sessionaware recommendation are consistently the most accurate recommenders. Lastly, a summary of the different challenges in the VoD domain is proposed, discussing on the best algorithmic solutions found, and proposing future research directions to be conducted based on the results obtained.

show abstract

ContentWise Impressions

Cited by 16 publications

References 18 publications

A Methodology for the Offline Evaluation of Recommender Systems in a User Interface with Multiple Carousels

A Methodology for the Offline Evaluation of Recommender Systems in a User Interface with Multiple Carousels

Offline recommender system evaluation: Challenges and new directions

From Data Analysis to Intent-Based Recommendation: An Industrial Case Study in the Video Domain

Contact Info

Product

Resources

About