Detecting noise in recommender system databases

Abstract

In this paper, we propose a framework that enables the detection of noise in recommender system databases. We consider two classes of noise: natural and malicious noise. The issue of natural noise arises from imperfect user behaviour (e.g. erroneous/careless preference selection) and the various rating collection processes that are employed. Malicious noise concerns the deliberate attempt to bias system output in some particular manner. We argue that both classes of noise are important and can adversely effect recommendation performance. Our objective is to devise techniques that enable system administrators to identify and remove from the recommendation process any such noise that is present in the data. We provide an empirical evaluation of our approach and demonstrate that it is successful with respect to key performance indicators.

I Like It... I Like It Not: Evaluating User Ratings Noise in Recommender Systems

“…Mahony et al [13] classify noise in RS into natural and malicious. The former refers to the definition of user generated noise provided in this paper, while the latter refers to noise that is deliberately introduced in a system in order to bias the results.…”

Managing Natural Noise in Recommender Systems

“…Specifically, Ekstrand et al [11] pointed out that the rating elicitation process is not error-free, hence the ratings can contain noise. They mentioned that such a noise, previously coined natural noise (NN) in [12], could be caused by human error, mixing of factors in the rating process, uncertainty and other factors. They stated that its detection and correction should provide more accurate recommendations.…”

“…Several authors have pointed out that the user preferences in RS could be inconsistent due to several reasons [12]. These inconsistencies have been classified in two main groups: (i) malicious noise, when the inconsistencies are intentionally inserted to bias the recommendation [21], or (ii) natural noise, when the inconsistencies appear without malicious intentions [22].…”

“…The concept of natural noise was introduced by O'Mahony et al [12], in which the authors focus on discovering noisy ratings, both malicious and natural noise. Amatriain et al [22] performed a user study to obtain a better understanding about how natural noise tends to appear.…”

“…All previous methods manage natural noise in CFRS in different ways, for instance O'Mahony et al [12] and Li et al [8] remove noisy information from the dataset. Others use additional information beyond the rating matrix, such as Pham and Jung [9] and Amatriain et al [25].…”