MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer

Chalkidis, Ilias; Fergadiotis, Manos; Androutsopoulos, Ion

doi:10.48550/arxiv.2109.00904

Cited by 4 publications

(5 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Link ECHR [78] 11,000 116 US Law [79] 7,800 328 EU Law [80] 65,000 492 Contracts [81] 80,000 62 Contracts [82] 9,414 3 Harvard Law case [77] 52,800 86 CaseHOLD [77] CaseLaw-BERT [77] Harvard Law case […”

Section: ) Court View Generation Datasetsmentioning

confidence: 99%

A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges

Cui,

Shen,

Wen

2023

IEEE Access

View full text Add to dashboard Cite

Legal judgment prediction (LJP) applies Natural Language Processing (NLP) techniques to predict judgment results based on fact descriptions automatically. The present work addresses the growing interest in the application of NLP techniques to the task of LJP. Despite the current performance gap between machines and humans, promising results have been achieved in a variety of benchmark datasets, owing to recent advances in NLP research and the availability of large-scale public datasets. To provide a comprehensive survey of existing LJP tasks, datasets, models, and evaluations, this study presents the following contributions: (1) an analysis of 43 LJP datasets constructed in 9 different languages, together with a classification method of LJP based on three different attributes; (2) a summary of 16 evaluation metrics categorized into 4 different types to evaluate the performance of LJP models for different outputs;(3) a review of 8 legal-domain pretrained models in 4 languages, highlighting four major research directions for LJP; (4) state-of-the-art results for 11 representative datasets from different court cases and an in-depth discussion of the open challenges in this area. This study aims to provide a comprehensive review for NLP researchers and legal professionals to understand the advances in LJP over the past years, and to facilitate further joint efforts towards improving the performance of LJP models.

show abstract

Section: ) Court View Generation Datasetsmentioning

confidence: 99%

A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges

Cui,

Shen,

Wen

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Link ECHR [63] 1,1000 116 US Law [64] 7,800 328 EU Law [65] 65,000 492 Contracts [66] 80,000 62 Contracts [67] 9,414 3 Harvard Law case [62] 52,800 86 CaseHOLD [62] CaseLaw-BERT [62] Harvard Law case […”

Section: Charge Prediction Datasetsmentioning

confidence: 99%

A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges

Cui¹,

Shen²,

Nie³

et al. 2022

Preprint

View full text Add to dashboard Cite

Legal judgment prediction (LJP) applies Natural Language Processing (NLP) techniques to predict judgment results based on fact descriptions automatically. Recently, large-scale public datasets and advances in NLP research have led to increasing interest in LJP. Despite a clear gap between machine and human performance, impressive results have been achieved in various benchmark datasets. In this paper, to address the current lack of comprehensive survey of existing LJP tasks, datasets, models and evaluations, (1) we analyze 31 LJP datasets in 6 languages, present their construction process and define a classification method of LJP with 3 different attributes;(2) we summarize 14 evaluation metrics under four categories for different outputs of LJP tasks; (3) we review 12 legal-domain pretrained models in 3 languages and highlight 3 major research directions for LJP; (4) we show the state-of-art results for 8 representative datasets from different court cases and discuss the open challenges. This paper can provide up-to-date and comprehensive reviews to help readers understand the status of LJP. We hope to facilitate both NLP researchers and legal professionals for further joint efforts in this problem.

show abstract

“…While there can be up to hundreds of entries in a docket, the writers are required to whittle the long list down to typically a dozen or fewer of the documents most essential for understanding the case, which constitutes the source documents in Multi-LexSum. 7 Table 6 describes the rubric for whether to include a document based on its type.…”

Section: B Multi-lexsum Summary Writing and Reviewing Guidelines B1 R...mentioning

confidence: 99%

“…NLP has been applied to a variety of legal document types, including patents [52], legal provisions and contracts [38,49,54], legislative bills [34], and court documents [21,42,62]. The NLP tasks studied in this work range from document/sentence classification [5,7,54] to information extraction [4,25], question answering [30,32,49,63], and-most relevant to our work-automatic summarization [21,29,34,52]. As found in other specialized domains of language, legal NLP systems often benefit from starting from a large language model pre-trained on legal text [6,57,62].…”

Section: Introductionmentioning

confidence: 99%

Multi-LexSum: Real-World Summaries of Civil Rights Lawsuits at Multiple Granularities

Shen¹,

Lo²,

Yu³

et al. 2022

Preprint

View full text Add to dashboard Cite

With the advent of large language models, methods for abstractive summarization have made great strides, creating potential for use in applications to aid knowledge workers processing unwieldy document collections. One such setting is the Civil Rights Litigation Clearinghouse (CRLC), 1 which posts information about large-scale civil rights lawsuits, serving lawyers, scholars, and the general public. Today, summarization in the CRLC requires extensive training of lawyers and law students who spend hours per case understanding multiple relevant documents in order to produce high-quality summaries of key events and outcomes. Motivated by this ongoing real-world summarization effort, we introduce Multi-LexSum, a collection of 9,280 expert-authored summaries drawn from ongoing CRLC writing. Multi-LexSum presents a challenging multi-document summarization task given the length of the source documents, often exceeding two hundred pages per case. Furthermore, Multi-LexSum is distinct from other datasets in its multiple target summaries, each at a different granularity (ranging from one-sentence "extreme" summaries to multi-paragraph narrations of over five hundred words). We present extensive analysis demonstrating that despite the high-quality summaries in the training data (adhering to strict content and style guidelines), state-of-the-art summarization models perform poorly on this task. We release Multi-LexSum for further research in summarization methods as well as to facilitate development of applications to assist in the CRLC's mission. 2 Preprint. Under review.

show abstract

MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer

Cited by 4 publications

References 28 publications

A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges

A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges

A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges

Multi-LexSum: Real-World Summaries of Civil Rights Lawsuits at Multiple Granularities

Contact Info

Product

Resources

About