A general approach for predicting the behavior of the Supreme Court of the United States

Abstract

Building on developments in machine learning and prior work in the science of judicial prediction, we construct a model designed to predict the behavior of the Supreme Court of the United States in a generalized, out-of-sample context. To do so, we develop a time-evolving random forest classifier that leverages unique feature engineering to predict more than 240,000 justice votes and 28,000 cases outcomes over nearly two centuries (1816-2015). Using only data available prior to decision, our model outperforms null (baseline) models at both the justice and case level under both parametric and non-parametric tests. Over nearly two centuries, we achieve 70.2% accuracy at the case outcome level and 71.9% at the justice vote level. More recently, over the past century, we outperform an in-sample optimized null model by nearly 5%. Our performance is consistent with, and improves on the general level of prediction demonstrated by prior work; however, our model is distinctive because it can be applied out-of-sample to the entire past and future of the Court, not a single term. Our results represent an important advance for the science of quantitative legal prediction and portend a range of other potential applications.

Measuring the Content of Presidential Policy Making: Applying Text Analysis to Executive Branch Directives

“…A common use of text data in the study of the Supreme Court is decision forecasting with supervised learners, using either raw oral argument transcripts (Katz, Bommarito, and Blackman 2017;Kaufman, Kraft, and Sen 2019) or transcripts plus recorded audio (Dietrich, Enos, and Sen 2019). 4 This literature, though valuable, is less rooted in theoretical debates.…”

Computational geometry and the U.S. Supreme Court

“…It is worth clarifying, however, that the models studied in this paper do not offer a method for predicting Supreme Court outcomes (for that one could see [9,10]), nor do they attempt to explain why the justices voted the way they did in any particular case. Instead, the main point in these models is to try to understand why certain majority/minority devisions that appear disordered and confounding from a traditional liberal-to-conservative perspective (or even from a pragmaticto-legalistic perspective as in [8]) may perhaps be better viewed through the lens of the outcome spheres introduced in this paper and the consequent notion of ideological common ground revealed through the computational geometry methods discussed above.…”

Crowdsourcing Accurately and Robustly Predicts Supreme Court Decisions

“…37,38,39,40,41,42 Of greatest relevance to the current inquiry is the growing number of papers focused on predicting the actions of legal institutions and actors. 43,44,45,46 Given the overall significance of legal decisions not only to individual participants but also to capital markets 47 and society as a whole, 48,49 the development of rigorous quantitative legal predictions 50 can have wide-spread benefits. In existing literature, scholars have typically applied well-known algorithms or variants thereof to forecast the decisionmaking processes of legal actors and institutions.…”

“…In existing literature, scholars have typically applied well-known algorithms or variants thereof to forecast the decisionmaking processes of legal actors and institutions. 43,46 In addition, the literature has examined the performance of certain subject matter experts and compared their performance to statistical models. 46,51 Algorithmic approaches have in some cases matched or exceeded the accuracy of subject matter experts.…”