The Threat of Algocracy: Reality, Resistance and Accommodation

Abstract

EnhancementOne of the most noticeable trends in recent years has been the increasing reliance of public decision-making processes (bureaucratic, legislative and legal) on algorithms, i.e. computer programmed step-by-step instructions for taking a given set of inputs and producing an output. The question raised by this article is whether the rise of such algorithmic governance creates problems for the moral or political legitimacy of our public decision-making processes. Ignoring common concerns with data protection and privacy, it is argued that algorithm-driven decisionmaking does pose a significant threat to the legitimacy of such processes. Modeling my argument on Estlund's threat of epistocracy, I call this the 'threat of algocracy'. The article clarifies the nature of this threat, and addresses two possible solutions (named, respectively, "resistance" and "accommodation"). It is argued that neither solution is likely to be successful, at least not without risking many other things we value about social decision-making. The result is a somewhat pessimistic conclusion in which we confront the possibility that we are creating decision-making processes that constrain and limit opportunities for human participation. Abstract Abstract Introduction

Why Internal Moral Enhancement Might Be politically Better than External Moral Enhancement

“…If we took a purely instrumentalist approach to the legitimation of such decision-making, we would presumably be happy if the legislative body reached the best possible moral outcome in relation to those issues. 32 It would not matter exactly how this outcome was reached. It wouldn't matter whether the methods bypassed conscious moral reasoning or not.…”

“…Even if the devices are transparent and understandable by human users, they often rely for their utility on well-known psychological biases and heuristics. Where a device is used to outsource the hard labour of moral reasoning and judgment, people are likely to grow complacent, tending to trust 33 On the question of what is required for true participativeness, see Machin [x] 34 For a lengthy defence of this view see Danaher [32]; also Burrell [41] 35 On these problems, see Kitchin [42], Burrell [41] and Danaher [32] 36 For a discussion of this black-boxing effect, see Pasquale [24], Burrell [41], and Danaher [32] the judgment of the technology over their own. They are also likely to be prone to degeneration effects.…”

The ghost in the legal machine: algorithmic governmentality, economy, and the practice of law

“…As Danaher (2016) notes, algorithms may not only remove human input from the private sector, but that there is a threat of algocracy -whereby algorithm-based systems could come to "structure and constrain the opportunities for human participation in, and comprehension of, public decision making." (p.245).…”

Working Online: An Exploration of Social Recruiting and Digital Marketplaces

“…The fetishism for reputation acquisition that broadly emerges in the various examples here discussed should alert us on the risks of what may be labelled a potential 'algocracy'. 16 This term identifi es the potentially dystopian consequence of what I have also called elsewhere 17 a 'Klout culture'-that is, the principle for which online algorithms and especially reputation metrics are uncritically approached as objective sources of information upon which to found supposedly unbiased evaluation of brands, individuals and activities of various sorts. As devices for reputation measurement proliferate, a culture that promotes an uncritical approach to metrics seems on the rise and confl icts with the risky aspects that these practices possess-fi rst and foremost, the generally undisclosed criteria upon which these algorithms base their calculations.…”