COLIEE-2018: Evaluation of the Competition on Legal Information Extraction and Entailment

Kano, Yoshinobu; Kim, Miyoung; Yoshioka, Masaharu; Lu, Yao; Rabelo, Juliano; Kiyota, Naoki; Goebel, Randy; Satoh, Ken

doi:10.1007/978-3-030-31605-1_14

Cited by 34 publications

(25 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are several LegalIR datasets, including COLIEE (Kano et al, 2018), CaseLaw (Locke and Zuccon, 2018), and CM (Xiao et al, 2019). Both COLIEE and CaseLaw are involved in retrieving most relevant articles from a large corpus, while data examples in CM give three legal documents for calculating similarity.…”

Section: Related Workmentioning

confidence: 99%

“…Duan et al (2019) propose CJRC, a legal reading comprehension dataset with the same format as SQUAD 2.0 (Rajpurkar et al, 2018), which includes span extraction, yes/no questions, and unanswerable questions. Besides, COLIEE (Kano et al, 2018) contains about 500 yes/no questions. Moreover, the bar exam is a professional qualification examination for lawyers, so bar exam datasets (Fawei et al, 2016;Zhong et al, 2019a) may be quite hard as they require professional legal knowledge and skills.…”

Section: Related Workmentioning

confidence: 99%

“…In recent years, with rapid developments in deep learning, researchers begin to apply deep learning techniques to LegalAI. Several new LegalAI datasets have been proposed (Kano et al, 2018;Duan et al, 2019;Chalkidis et al, 2019b,a), which can serve as benchmarks for research in the field. Based on these datasets, researchers began exploring NLP-based solutions to a variety of LegalAI tasks, such as Legal Judgment Prediction (Aletras et al, 2016;Luo et al, 2017;Chen et al, 2019), Court View Generation (Ye et al, 2018), Legal Entity Recognition and Classification (Cardellino et al, 2017;ANGELIDIS et al, 2018), Legal Question Answering (Monroy et al, 2009;Taniguchi and Kano, 2016;Kim and Goebel, 2017), Legal Summarization (Hachey and Grover, 2006;Bhattacharya et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence

Zhong¹,

Xiao²,

Tu³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

197

View full text Add to dashboard Cite

Legal Artificial Intelligence (LegalAI) focuses on applying the technology of artificial intelligence, especially natural language processing, to benefit tasks in the legal domain. In recent years, LegalAI has drawn increasing attention rapidly from both AI researchers and legal professionals, as LegalAI is beneficial to the legal system for liberating legal professionals from a maze of paperwork. Legal professionals often think about how to solve tasks from rulebased and symbol-based methods, while NLP researchers concentrate more on data-driven and embedding methods. In this paper, we describe the history, the current state, and the future directions of research in LegalAI. We illustrate the tasks from the perspectives of legal professionals and NLP researchers and show several representative applications in LegalAI. We conduct experiments and provide an indepth analysis of the advantages and disadvantages of existing works to explore possible future directions. You can find the implementation of our work from https://github. com/thunlp/CLAIM.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence

Zhong¹,

Xiao²,

Tu³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

197

View full text Add to dashboard Cite

show abstract

“…The specic task of assessing textual entailment for case law documents is quite new. The rst COLIEE edition which included this task was in 2018 [18], and the two best performing approaches are described below.…”

Section: Case Law Textual Entailmentmentioning

confidence: 99%

Combining Similarity and Transformer Methods for Case Law Entailment

Rabelo

Kim

Goebel

2019

Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law

Self Cite

View full text Add to dashboard Cite

We tackle the complex problem of determining entailment relationships between case law documents, one of the tasks in the Competition on Legal Information Extraction and Entailment (COLIEE). With input of an entailed fragment from a case coupled with a candidate entailing paragraph from a noticed case, our approach relies on four main components: (1) extraction of similarity measures between the two pieces of text; (2) application of a transformer-based technique on the input text; (3) applying a threshold-based classier; and (4) post-processing the results considering the a priori probability determined by the data distribution on the training samples and combining the results of (1) and (2). Our experiments achieved an F-score of 0.70 on the ocial COLIEE test dataset, ranking rst among all competitors for that task in the 2019 competition.

show abstract

“…The Competition on Legal Information Extraction/Entailment, for the first time in 2018, organized a legal case retrieval task [8]. The task was tackled in various approaches in both lexical matching and deep learning methods.…”

mentioning

confidence: 99%

Building Legal Case Retrieval Systems with Lexical Matching and Summarization using A Pre-Trained Phrase Scoring Model

Tran

Nguyen

Satoh

2019

Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law

Self Cite

View full text Add to dashboard Cite

We present our method for tackling the legal case retrieval task of the Competition on Legal Information Extraction/Entailment 2019. Our approach is based on the idea that summarization is important for retrieval. On one hand, we adopt a summarization based model called encoded summarization which encodes a given document into continuous vector space which embeds the summary properties of the document. We utilize the resource of COLIEE 2018 on which we train the document representation model. On the other hand, we extract lexical features on different parts of a given query and its candidates. We observe that by comparing different parts of the query and its candidates, we can achieve better performance. Furthermore, the combination of the lexical features with latent features by the summarization-based method achieves even better performance. We have achieved the state-of-the-art result for the task on the benchmark of the competition.

show abstract

COLIEE-2018: Evaluation of the Competition on Legal Information Extraction and Entailment

Cited by 34 publications

References 4 publications

How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence

How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence

Combining Similarity and Transformer Methods for Case Law Entailment

Building Legal Case Retrieval Systems with Lexical Matching and Summarization using A Pre-Trained Phrase Scoring Model

Contact Info

Product

Resources

About