Pooling decisions decreases variation in response bias and accuracy

Kurvers, Ralf H. J. M.; Herzog, Stefan M.; Hertwig, Ralph; Krause, Jens; Wolf, Max

doi:10.1016/j.isci.2021.102740

Cited by 10 publications

(9 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the following, we discuss three contributions the present approach makes to research on within-expert inconsistency. First, in order to reduce inconsistency and thus improve accuracy, previous perspectives suggest using interventions that increase the reliability of information processing, such as reducing the amount of information presented 39 , decomposing a complex task into smaller ones 40 , or combining an individual’s repeated judgments 41 – 43 or judgments from different individuals 44 . Our work suggests a complementary approach to improving accuracy in the face of unreliability—namely, encouraging experts to make a second assessment whenever they are not confident in their initial decision and to then apply the confidence rule across the two decisions.…”

Section: Discussionmentioning

confidence: 99%

How experts’ own inconsistency relates to their confidence and between-expert disagreement

Litvinova

Kurvers

Hertwig

et al. 2022

Sci Rep

Self Cite

View full text Add to dashboard Cite

People routinely rely on experts’ advice to guide their decisions. However, experts are known to make inconsistent judgments when judging the same case twice. Previous research on expert inconsistency has largely focused on individual or situational factors; here we focus directly on the cases themselves. First, using a theoretical model, we study how within-expert inconsistency and confidence are related to how strongly experts agree on a case. Second, we empirically test the model’s predictions in two real-world datasets with a diagnostic ground truth from follow-up research: diagnosticians rating the same mammograms or images of the lower spine twice. Our modeling and empirical analyses converge on the same novel results: The more experts disagree in their initial decisions about a case (i.e., as consensus decreases), the less confident individual experts are in their initial decision—despite not knowing the level of consensus—and the more likely they are to judge that same case differently when facing it again months later, regardless of whether the expert consensus is correct. Our results suggest the following advice when faced with two conflicting decisions from a single expert: In the absence of more predictive cues, choose the more confident decision.

show abstract

Section: Discussionmentioning

confidence: 99%

How experts’ own inconsistency relates to their confidence and between-expert disagreement

Litvinova

Kurvers

Hertwig

et al. 2022

Sci Rep

Self Cite

View full text Add to dashboard Cite

show abstract

“…A notable instance in several animal groups is “the wisdom of crowds”, where individuals can improve estimation accuracy by aggregating their separate estimates of the environmental state. Examples of this phenomenon range from nest-site choices by ant colonies (58, 59), to foraging decisions by fish schools (60) and even medical diagnostics (61, 62). Although bacteria do not possess the complex sensing, cognition and feedback mechanisms found in social animals and humans, our work shows that the same collective functionality can arise in populations of bacteria with simple gene regulation networks and could have driven the evolution of quorum sensing as one of the most widely used communication systems in bacteria.…”

Section: Discussionmentioning

confidence: 99%

Quorum sensing as a mechanism to harness the wisdom of the crowds

Moreno-Gámez

Hochberg

Doorn

2022

Preprint

View full text Add to dashboard Cite

Bacteria release and sense small molecules called autoinducers (AIs) in a process known as quorum sensing (QS). The prevailing interpretation of QS is that by sensing AI concentrations, bacteria estimate population density to regulate the expression of functions that are only beneficial when carried out by a sufficiently large number of cells. However, a major challenge to this interpretation is that the concentration of AIs strongly depends on the environment, often rendering AI-based estimates of cell density unreliable. Here we propose an alternative interpretation of QS, where bacteria, by releasing and sensing AIs, harness social interactions to sense the environment as a collective. As shown by a model, this functionality can explain the evolution of QS, and results from individuals improving their estimation accuracy by pooling many imperfect estimates — analogous to the 'wisdom of the crowds' in decision theory. Importantly, our model reconciles the observed dependence of QS on both population density and the environment and explains why several QS systems regulate the production of private goods.

show abstract

“…It could be seen, for example, in the recent case of Xavier Lucas v. France, where the decision of the court was recognised as overly formalistic, bearing in mind that the applicant could not send e-files because there were practical hurdles. 11 This example clearly illustrates that the courts are capable of remaining a bulwark of substantive justice. The requirement to use electronic means, however, is becoming more and more common.…”

Section: The Rule Of Law In Times Of Digital Technologiesmentioning

confidence: 97%

Algorithms in the Courts: Is There Any Room for a Rule of Law?

2022

AJEE

View full text Add to dashboard Cite

The rule of law is one of the fundamental pillars, along with human rights and democracy, which are affected by digitalisation today. Digital technologies used for the victory of populism, the manipulation of opinions, attacks on the independence of judges, and the general instrumentalisation of the law contribute significantly to the onset of negative consequences for the rule of law. Particularly dangerous are the far-reaching consequences of the algorithmisation of decision-making, including judicial decisions. The theoretical line of this research is based on the axiological method since the rule of law, democracy, and human rights are not only the foundations of legal order, but also values recognised in many societies and supported at the individual level. The study also relied on the phenomenological method in terms of assessing the experience of being influenced by digital technologies in public and private life. The practical line of research is based on the analysis of cases of the European Court of Human Rights and the Court of Justice to illustrate the changes in jurisprudence influenced by digitalisation. This article argues that the potential weakening of the rule of law could be related to the impact of certain technologies itself, and to their impact on certain values and foundations which is significantly aggravated. Judicial independence is affected since the judges are involved in digital interactions and are influenced by technologies along personal and public lines. That technologies often belong private sector but are perceived as neutral and infallible, which is highly predictive of court decisions. This leads to a distortion of the essence of legal certainty and a shift of trust from the courts to certain technologies and their creators. The possibility of algorithmic decision-making raises the question of whether the results will be fairer, or at least as fair, as those handed down by human judges. This entails two problems, the first of which is related to the task of interpreting the law and the second of which involves the need to explain decisions. Algorithms, often perceived as reliable, are not really capable of interpreting the law, and their ability to provide proper explanations for decisions or understand context and social practices is questionable. Even partial reliance on algorithms should be limited, given the growing inability to draw a line between the human and algorithmic roles in decision-making and determine who should be responsible for the decision and to what extent.

show abstract

Pooling decisions decreases variation in response bias and accuracy

Cited by 10 publications

References 58 publications

How experts’ own inconsistency relates to their confidence and between-expert disagreement

How experts’ own inconsistency relates to their confidence and between-expert disagreement

Quorum sensing as a mechanism to harness the wisdom of the crowds

Algorithms in the Courts: Is There Any Room for a Rule of Law?

Contact Info

Product

Resources

About