Extracting Outcomes from Appellate Decisions in US State Courts

Abstract

Predicting the outcome of a legal process has recently gained considerable research attention. Numerous attempts have been made to predict the exact outcome, judgment, charge, and fines of a case given the textual description of its facts and metadata. However, most of the effort has been focused on Chinese and European law, for which there exist annotated datasets. In this paper, we introduce CASELAW4 — a new dataset of 350k common law judicial decisions from the U.S. Caselaw Access Project, of which 250k have been automatically annotated with binary outcome labels of AFFIRM or REVERSE by our hybrid learning system. To our knowledge, it is the first attempt to perform outcome extraction (a) on such a large volume of English-language judicial opinions, (b) on the Caselaw Access Project data, and (c) on US State Courts of Appeal cases, and it paves the way to large-scale outcome prediction and advanced legal analytics using U.S. Case Law. We set up baseline results for the outcome extraction task on the new dataset, achieving an F-measure of 82.32%.

Automatically Identifying Eviction Cases and Outcomes Within Case Law of Dutch Courts of First Instance

“…The performance on the test set can be found in Tables 10 and 11. 9 The following command, showing all used parameters, was used to fit our final model:…”

“…We have built a Linear SVC that uses character (1-7) n-grams, and optimised it for a number of other parameters. 9 The results of the model during the cross-validation stage can be found in Tables 8 and 9. 9.…”

Rethinking the field of automatic prediction of court decisions

“…Outcome identification falls under the field of information extraction and when not confused with predicting court decisions is often also referred to as outcome extraction (e.g., Petrova et al 2020). Given the growing body of published case law across the world, the automation of this task may be very useful, since many courts publish case law without any structured information (i.e.…”

“…Automation of outcome identification allows one to save time when collecting this information. While the task is not necessarily always trivial for a machine and depends on how the verdict is formulated (see, for instance, Vacek and Schilder (2017), Petrova et al (2020) and Tagny-Ngompé et al ( 2020)), there is nonetheless an expectation that these automated systems should achieve (almost) perfect performance to justify the automation. However, the approach to outcome identification is highly dependent on the structure of judgements in a particular legal domain or jurisdiction and the language of the case law.…”

Lex Rosetta

“…Examples from several domains and countries include administrative decisions from the U.S. [33,41], multi-domain court decisions from India [6], international arbitration decisions [9], or even multi-{domain,country} adjudicatory decisions in English [28]. Identifying a section that states an outcome of the case has also received considerable attention separately [25,38]. To the best of our knowledge, existing work on functional segmentation of court decisions is limited to a single language-ours being the first paper exploring the task jointly on legal documents in multiple languages.…”