Crowdvoting Judgment: An Analysis of Modern Peer Review

Abstract

In this paper, we propose and analyze models of self policing in online communities, in which assessment activities, typically handled by firm employees, are shifted to the “crowd.” Our underlying objective is to maximize firm value by maintaining the quality of the online community to prevent attrition, which, given a parsimonious model of voter participation, we show can be achieved by efficiently utilizing the crowd of volunteer voters. To do so, we focus on minimizing the number of voters needed for each assessment, subject to service-level constraints, which depends on a voting aggregation rule. We focus our attention on classes of voting aggregators that are simple, interpretable, and implementable, which increases the chance of adoption in practice. We consider static and dynamic variants of simple majority-rule voting, with which each vote is treated equally. We also study static and dynamic variants of a more sophisticated voting rule that allows more accurate voters to have a larger influence in determining the aggregate decision. We consider both independent and correlated voters and show that correlation is detrimental to performance. Finally, we take a system view and characterize the limit of a costless crowdvoting system that relies solely on volunteer voters. If this limit does not satisfy target service levels, then costly firm employees are needed to supplement the crowd.

Evaluating blockchain as a participatory organisational system: looking for transaction efficiency

“…In this way, the accuracy of the evaluator is underestimated, and more evaluators are needed for each project. However, evaluators should not be eliminated due to lack of reliability, contributing to the collective evaluation assuming the minimum training that each one can present to 95%, the blockchain requires a greater number of evaluators to neutralise this supposed lack of expertise; for more details see Wagner (2020).…”

Socially Responsible Crowdsourcing With Small Businesses in the Covid-19 Era: An Empirical Study

“…In small businesses, its use is to evaluate a product for quality, price, and timeliness through votes received from the public. This in turn can be used to modify the product for wider acceptance by consumers, with the added benefit of gaining goodwill and enhancing the value of the firm (Wagner, 2020). Crowdvoting thus employs multiple users' judgment to evaluate content.…”