Justice Is Less Blind, and Less Legalistic, than We Thought: Evidence from an Experiment with Real Judges

Abstract

We experimentally investigate the determinants of judicial decisions in a setting resembling real-world judicial decision-making. U.S. federal judges (N=32) spend 55 minutes judging a real appeals case from an international tribunal, with minor modifications to accommodate the experimental treatments. The fictitious briefs focus on one easily understandable issue of law. Our 2×2 between-subject factorial design crosses a weak precedent and legally irrelevant defendant characteristics. In a survey, law professors predicted that the precedent would have a stronger effect than the defendant characteristics. In actuality, the precedent has no detectable effect on the judges' decisions, whereas the two defendants' affirmance rates differ by 45% (p<.01). Judges' written reasons, on the other hand, do not mention defendant characteristics at all, focusing instead on the precedent and other legalistic and policy considerations.

Do judges hate speculators?

“…This result is in line with that of Redding and Repucci (1999). • Spamann and Klöhn (2016) created a virtual file of an international criminal law appeal case with briefs, precedents, and a trial judgment displayed on a tablet computer. They gave a group of 32 U.S. federal judges 55 min to read the file and decide the case.…”

“…In summary, the question whether judges' professionalism shields them from the influence of legally irrelevant decision determinants is far from answered. While Kahan et al (2016) provide evidence that gives reason for optimism, the results of Wistrich et al's (2015) and Spamann and Klöhn's (2016) studies are rather sobering. We add to this literature by providing another piece of evidence largely in line with Kahan et al (2016).…”

Inputs and Outputs on Appeal: An Empirical Study of Briefs, Big Law, and Case Complexity

“…Even large datasets may not give adequate insight into causation and experiments present additional opportunities for learning. Compared to standard vignette studies, some researchers are creating somewhat more realistic decision environments for judge participants (Spamann & Klöhn 2016). Simulated documents can be randomly assigned to judges, who are given extensive time to make judgments.…”

Law Matters -- Less Than We Thought

“…Participants at the workshop and thus in the experiment included circuit, district, bankruptcy, and magistrate judges, and one judge from the Court of Federal Claims, whom we grouped with magistrate judges in our analysis to preserve anonymity. Spamann and Klöhn (2016) conducted their experiment in 2015 at the same program.…”

“…That is, the empirical literature on judicial decision-making to date has been focused on the effects of non-law. The only experiment with judicial subjects that directly tested the effect of law found no effect (Spamann and Klöhn 2016). Nevertheless, the legal variation in Spamann and Klöhn (2016) (a precedent) was mild, and the experimental setting (international law) was unfamiliar to most of the participants (U.S. federal judges).…”