On the inaccuracy of numerical ratings: dealing with biased opinions in social networks

Abstract

In this work, we study the potential problems emanating from using numerical ratings in social networks to rank entities regarding their reputation. In particular, we empirically demonstrate how reputation rankings as collected and managed by current systems are likely to be skewed due to subjectivity problems associated with the use of numerical ratings to encapsulate preferences. With the aim of overcoming these problems, we put forward an approach in which users are asked for their opinions about entities in a comparative fashion. In order to select the most appropriate users to be queried, we take advantage of the social structure derived from the interactions among users and entities following a principle of heterogeneity. Finally, we evaluate the proposed approach in the domain of movie ratings by using real datasets collected from different web sites.

Advances in Social Media Research: Past, Present and Future

“…Hildebrand et al (2013) use data from a European car manufacturer allowing self-designed products to reveal that feedback from other community members lessens uniqueness whilst increasing dissatisfaction. Centeno et al (2015) address the skewed reputation rankings problem in movie ratings by suggesting the use of comparative user opinions. Ma et al (2013) analyse data from Yelp to test bias in online reviews and find that frequent and longer reviews successfully combat such biases.…”

Towards learning resources rankings in MOOCs: A pairwise based reputation mechanism

“…This refers to the underlying behavioral phenomenon, which leads to average ratings often showing a heavily and usually positive skewed tendency. This problem is stated in [12] where opinions given by users and critics about movies are compared. Whilst the distribution of critics ratings have a much closer fit to a normal distribution, ratings given by users present a clearly positive bias.…”

“…In order to deal with biased opinions and their potential bias problems, we have developed a novel approach [12] where preferences of users are captured in a comparative fashion. In this work we analyze and state that such a mechanism can be applied for estimating more effectively the reputation of learning resources uploaded in MOOCS.…”

“…For example, in [12] the authors prove that the use of numerical ratings leads to important shortcomings when aggregating opinions, due to subjectivity, may entail misinterpretations in the global opinions.…”

Special issue on agreement technologies

“…In the article entitled BOn the inaccuracy of numerical ratings: dealing with biased opinions in social networks^, Centeno et al (2015), deal with the subjectivity issues concerning numerical rankings of entities' reputation in social networks. Their proposal concerns gathering comparative opinions about entities, from users.…”