No abstract

No abstract

This paper describes the results of a study conducted in February 2013 on Amazon Mechanical Turk aimed at identifying various determinants of credibility evaluations. 2046 adult participants evaluated credibility of websites with diversified trustworthiness reference index. We concentrated on psychological factors that lead to the characteristic positive bias observed in many working social feedback systems on the Internet. We have used International Personality Item Pool (IPIP) and measured the following traits: trust, conformity, risk taking, need for cognition and intellect. Results suggest that trustworthiness and risk taking are factors clearly differentiating people with respect to tendency to overestimate, underestimate and judge accordingly websites' credibility. Intuitively people characterized by high general trust tend to be more generous in their credibility evaluations. On the other hand, people who are more willing to take risk, tend to be more critical of the Internet content. The latter indicates that high credibility evaluations are being treated as a default option, and lower ratings require special conditions. Other, more detailed psychological patterns related to websites' credibility evaluations are described in full paper.

No abstract

Machine learning algorithms and recommender systems trained on human ratings are widely in use today. However, human ratings may be associated with a high level of uncertainty and are subjective, influenced by demographic or psychological factors. We propose a new approach to the design of object classes from human ratings: the use of entire distributions to construct classes. By avoiding aggregation for class definition, our approach loses no information and can deal with highly volatile or conflicting ratings. The approach is based the concept of the Earth Mover's Distance (EMD), a measure of distance for distributions. We evaluate the proposed approach based on four datasets obtained from diverse Web content or movie quality evaluation services or experiments. We show that clusters discovered in these datasets using the EMD measure are characterized by a consistent and simple interpretation. Quality classes defined using entire rating distributions can be fitted to clusters of distributions in the four datasets using two parameters, resulting in a good overall fit. We also consider the impact of the composition of small samples on the distributions that are the basis of our classification approach. We show that using distributions based on small samples of 10 evaluations is still robust to several demographic and psychological variables. This observation suggests that the proposed approach can be used in practice for quality evaluation, even for highly uncertain and subjective ratings.