Rating Effects on Social News Posts and Comments

Glenski, Maria; Weninger, Tim

doi:10.1145/2963104

Cited by 31 publications

(36 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Since most platforms reveal the aggregate vote thus far, the initial votes act as a social signal to influence the subsequent voters, forming the basis of social influence bias. We know from prior work that for platforms that reveal social signal, users exhibit a herding effect [23]: the first few votes on content can unduly skew the subsequent votes. Consider a counterfactual scenario, where two "identical" content initially receive dissimilar votes; then, social influence bias implies that the content with higher aggregate vote thus far will receive more upvotes.…”

Section: Voter Biasmentioning

confidence: 99%

“…There is a plethora of research on detecting and quantifying voter biases in online platforms [51,38,34,43,36,33,59,28,54,57,9,1,23]. Broadly, researchers have adopted one of the following two approaches: 1) conduct experiments to create different voting conditions for studying participants [51,38,43,36,28,1,23]; 2) develop statistical models to analyze historical voting data [34,33,59,54,57,9]. Both approaches have limitations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Quantifying Voter Biases in Online Platforms

Dev

Karahalios

Sundaram

2019

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

In content-based online platforms, use of aggregate user feedback (say, the sum of votes) is commonplace as the "gold standard" for measuring content quality. Use of vote aggregates, however, is at odds with the existing empirical literature, which suggests that voters are susceptible to different biases-reputation (e.g., of the poster), social influence (e.g., votes thus far), and position (e.g., answer position). Our goal is to quantify, in an observational setting, the degree of these biases in online platforms. Specifically, what are the causal effects of different impression signals-such as the reputation of the contributing user, aggregate vote thus far, and position of content-on a participant's vote on content? We adopt an instrumental variable (IV) framework to answer this question. We identify a set of candidate instruments, carefully analyze their validity, and then use the valid instruments to reveal the effects of the impression signals on votes. Our empirical study using log data from Stack Exchange websites shows that the bias estimates from our IV approach differ from the bias estimates from the ordinary least squares (OLS) method. In particular, OLS underestimates reputation bias (1.6-2.2x for gold badges) and position bias (up to 1.9x for the initial position) and overestimates social influence bias (1.8-2.3x for initial votes). The implications of our work include: redesigning user interface to avoid voter biases; making changes to platforms' policy to mitigate voter biases; detecting other forms of biases in online platforms.

show abstract

Section: Voter Biasmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Quantifying Voter Biases in Online Platforms

Dev

Karahalios

Sundaram

2019

Proc. ACM Hum.-Comput. Interact.

View full text Add to dashboard Cite

show abstract

“…The voting pattern in the Reddit [14] has been studied to analyze the upvoting of posts from a new page to the front page and behavior of users towards some posts which are getting positive or negative votes.They have studied the posts mentioning Wikileaks and Fox News and to see the impact of negative voting on them, although working on only one month of data. One study related to rating effect on posts and comments [8] revealed that random rating manipulations on posts and comments led to significant changes in downstream ratings leading to significantly different final outcomes -positive herding effects for positive treatments on posts, increasing final ratings on the average, but not for positive treatments on comments, while negative herding effects for negative treatments on posts and comments, decreasing the final ratings on average. Another exploratory study [20] on the dynamics of discussion threads found topical hierarchy in discussion threads, and how it is possible to use them to enhance Web search.…”

Section: Related Workmentioning

confidence: 99%

Characterizing Behavioral Trends in a Community Driven Discussion Platform

Thukral¹,

Chatterjee²,

Meisheri³

et al. 2019

Lecture Notes in Social Networks

View full text Add to dashboard Cite

This article presents a systematic analysis of the patterns of behavior of individuals as well as groups observed in community-driven platforms for discussion like Reddit, where users usually exchange information and viewpoints on their topics of interest. We perform a statistical analysis of the behavior of posts and model the users' interactions around them. A platform like Reddit which has grown exponentially, starting from a very small community to one of the largest social networks, with its large user base and popularity harboring a variety of behavior of users in terms of their activity. Our work provides interesting insights about a huge number of inactive posts which fail to attract attention despite their authors exhibiting Cyborg-like behavior to attract attention. We also observe short-lived yet extremely active posts emulate a phenomenon like Mayfly Buzz. A method is presented, to study the activity around posts which are highly active, to determine the presence of Limelight hogging activity. We also present a systematic analysis to study the presence of controversies in posts. We analyzed data from two periods of one-year duration but separated by few years in time, to understand how social media has evolved through the years.

show abstract

“…Online, we must rely on ratings provided by previous customers instead. Online rating systems, however, suffer from the known problem of social influence, also termed herding, which expresses the fact that raters tend to be biased by the opinions of previous raters [2,6,8,10,13,17] and which can make online rating systems fickle and sensitive to small variations in early ratings: if the first few reviews of a product happen to swing a certain way (or are purposefully engineered that way in an act of review spamming [9]), this can unduly skew subsequent reviews. This can have severe implications for both customers, who may end up with unsatisfying products, and producers, whose highquality products may end up being bought less than they deserve.…”

Section: Introductionmentioning

confidence: 99%

When Sheep Shop

Lederrey

West

2018

Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18

View full text Add to dashboard Cite

As online shopping becomes ever more prevalent, customers rely increasingly on product rating websites for making purchase decisions. The reliability of online ratings, however, is potentially compromised by the so-called herding effect: when rating a product, customers may be biased to follow other customers' previous ratings of the same product. This is problematic because it skews long-term customer perception through haphazard early ratings. The study of herding poses methodological challenges. In particular, observational studies are impeded by the lack of counterfactuals: simply correlating early with subsequent ratings is insufficient because we cannot know what the subsequent ratings would have looked like had the first ratings been different. The methodology introduced here exploits a setting that comes close to an experiment, although it is purely observational-a natural experiment. Our key methodological device consists in studying the same product on two separate rating sites, focusing on products that received a high first rating on one site, and a low first rating on the other. This largely controls for confounds such as a product's inherent quality, advertising, and producer identity, and lets us isolate the effect of the first rating on subsequent ratings. In a case study, we focus on beers as products and jointly study two beer rating sites, but our method applies to any pair of sites across which products can be matched. We find clear evidence of herding in beer ratings. For instance, if a beer receives a very high first rating, its second rating is on average half a standard deviation higher, compared to a situation where the identical beer receives a very low first rating. Moreover, herding effects tend to last a long time and are noticeable even after 20 or more ratings. Our results have important implications for the design of better rating systems.

show abstract

Rating Effects on Social News Posts and Comments

Cited by 31 publications

References 64 publications

Quantifying Voter Biases in Online Platforms

Quantifying Voter Biases in Online Platforms

Characterizing Behavioral Trends in a Community Driven Discussion Platform

When Sheep Shop

Contact Info

Product

Resources

About