Automatic Charge Identification from Facts: A Few Sentence-Level Charge Annotations is All You Need

Abstract

Automatic Charge Identification (ACI) is the task of identifying the relevant legal charges given the facts of a situation and the statutory laws that define these charges, and is a crucial aspect of the judicial process. Prior works focus on learning charge-side representations by modeling relationships between the charges, but not much effort has been made in improving fact-side representations. We observe that only a small fraction of sentences in the facts actually indicates the charges. We show that by using a very small subset (< 3%) of fact descriptions annotated with sentence-level charges, we can achieve an improvement across a range of different ACI models, as compared to modeling just the main document-level task on a much larger dataset. Additionally, we propose a novel model that utilizes sentence-level charge labels as an auxiliary task, coupled with the main task of document-level charge identification in a multi-task learning framework. The proposed model comprehensively outperforms a large number of recent baseline models for ACI. The improvement in performance is particularly noticeable for the rare charges which are known to be especially challenging to identify.

A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges

“…Language Input Output Download Rationale Decision SwissJudgmentPrediction [26] German Pleas r Facts r -Plea judgments r Link Italian French Sulea et al [14] Facts r Others r -TSCC [40] Thai Facts a Articles a -Link JUSTICE [27] English Pleas r Facts r -Plea judgments r -BStricks_LDC [41] Facts r Others r Link ILDC [42] Fact snippets r Link ACI [43] Facts a,r Charges a Link echr [13] Facts r…”

“…The charge prediction [48] is interpreted based on the extraction of Phrase-level facts. Furthermore, the relationship between the sentence-level fact and its corresponding charges has been modeled to the document-level charge prediction [43].…”

“…In constructing datasets, there have been at least two lines of work, as seen in Table 2. The first one employs a machine to extract the necessary metadata, such as using Regular Expression to identify and mask judgment results ( [26,14,41,42]) or to extract the input-output data samples( [13,25,44,45,24,29,47,15,49,33,50,53]) from judgment documents of given court and applying the deep neural model to extract facts from Court judgment documents as [43] for evaluating the LJP model. The second line of work recruits legal experts to annotate rationale sentences [42,48] as gold sentences or to annotate legal domain knowledge (which are not included in the judgment documents, such as [46,51,52]).…”

“…RACP [48], has been constructed from CJO, containing 100,000 documents in which rationale sentences were annotated based on the extracted fact description and charge labels. Similarly, ACI [43] collected from the Supreme Court of India cases consists of 4,338 judgment documents with document-level charge label information facts were extracted by legal experts and automated method for 70 and 4,268 documents, and legal experts annotate sentence-level charges for 120 documents. Moreover, three datasets [51] have been published with selected case's fact part and extracted charges of judgment documents from CJO denoted as Criminal-S(small), Criminal-M(medium), Criminal-L(large).…”

“…Output Evaluation metrics ECtHR_crystal_ball [44] Articles Acc echr [13] MAMD [49] Charges CAIL-Long [47] Prison terms LD Articles MaF/MiF Charges SwissJudgmentPrediction [26] Plea judgments TSCC [40] Articles ACI [43] Charges MaF MaP MaR ECHR-CASES [25] Articles MLMN [46] DPAM [45] JS MaF RACP [48] Charges MaF Fact snippets Acc MaF QAjudge-CJO,QAjudge-PKU,QAjudge-CAIL [52] Elements answers MaF JUSTICE [27] Plea judgments…”