Investigating User Behavior in Legal Case Retrieval

Shao, Yunqiu; Wu, Yueyue; Liu, Yiqun; Mao, Jiaxin; Zhang, Min; Ma, Shaoping

doi:10.1145/3404835.3462876

Cited by 17 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Information Retrieval techniques are at the core of many legal research platforms, such as Lexis+ 2 and Westlaw Edge 3 , and have several critical application scenarios, such as legislative document retrieval [82], [83] and case retrieval [84], [85]. Depending on the application, the queries can be short (e.g., simple keywords), of medium length (e.g., Boolean or natural language queries) or long (e.g., whole documents).…”

Section: A An Overview Of Major Legal Nlp Tasksmentioning

confidence: 99%

“…Both PARM [96] and BERT-PLI [28] also tried to condense the documents, and performed paragraph-level modeling on top of candidates returned via BM25 or similar methods. Instead of focusing on the techniques, Shao et al [85] presented a comparative user behavior study between legal and general-domain search, and suggested that legal information retrieval is more challenging with respect to query length and number of clicks/pages, among other metrics. A recent survey on legal information retrieval is available here [97].…”

Section: A An Overview Of Major Legal Nlp Tasksmentioning

confidence: 99%

See 1 more Smart Citation

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

Song

Gao

et al. 2022

IEEE Access

View full text Add to dashboard Cite

We present the first comprehensive empirical evaluation of pre-trained language models (PLMs) for legal natural language processing (NLP) in order to examine their effectiveness in this domain.Our study covers eight representative and challenging legal datasets, ranging from 900 to 57K samples, across five NLP tasks: binary classification, multi-label classification, multiple choice question answering, summarization and information retrieval. We first run unsupervised, classical machine learning and/or non-PLM based deep learning methods on these datasets, and show that baseline systems' performance can be 4%∼35% lower than that of PLM-based methods. Next, we compare general-domain PLMs and those specifically pre-trained for the legal domain, and find that domain-specific PLMs demonstrate 1%∼5% higher performance than general-domain models, but only when the datasets are extremely close to the pretraining corpora. Finally, we evaluate six general-domain state-of-the-art systems, and show that they have limited generalizability to legal data, with performance gains from 0.1% to 1.2% over other PLM-based methods. Our experiments suggest that both general-domain and domain-specific PLM-based methods generally achieve better results than simpler methods on most tasks, with the exception of the retrieval task, where the best-performing baseline outperformed all PLM-based methods by at least 5%. Our findings can help legal NLP practitioners choose the appropriate methods for different tasks, and also shed light on potential future directions for legal NLP research.

show abstract

Section: A An Overview Of Major Legal Nlp Tasksmentioning

confidence: 99%

Section: A An Overview Of Major Legal Nlp Tasksmentioning

confidence: 99%

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

Song

Gao

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Many works tried to build a retrieval engine using links or citations between case documents and legal statutes [2,3,4]. Recently, Shao et al [22] and Liu et al [23] studied user behaviors and practical search interactions between users and systems and observed significant differences between legal case retrieval and general web search.…”

Section: B Legal Case Retrievalmentioning

confidence: 99%

Legal Element-oriented Modeling with Multi-view Contrastive Learning for Legal Case Retrieval

Wang

2022

2022 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

“…Given a case, the goal of the SCR task is to retrieve similar cases from the candidate pool according to the judgment criteria, and judges refer to similar cases to assist in the current case judgment. Previous SCR works [5,6,8,10,11,13,17,18,19,22,27] have achieved some success, [3,24] measure the similarity of cases using textual information, [4] develop similar case measurement strategies considering both subjective and objective evaluation, [22] attend legal elements for SCR through contrastive learning. However, two main challenges of existing SCR datasets have constrained the progress of the case retrieval models:…”

Section: Introductionmentioning

confidence: 99%

MUSER: A Multi-View Similar Case Retrieval Dataset

Li,

Hu,

Yao

et al. 2023

Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Similar case retrieval (SCR) is a representative legal AI application that plays a pivotal role in promoting judicial fairness. However, existing SCR datasets only focus on the fact description section when judging the similarity between cases, ignoring other valuable sections (e.g., the court's opinion) that can provide insightful reasoning process behind. Furthermore, the case similarities are typically measured solely by the textual semantics of the fact descriptions, which may fail to capture the full complexity of legal cases from the perspective of legal knowledge. In this work, we present MUSER, a similar case retrieval dataset based on multiview similarity measurement and comprehensive legal element with sentence-level legal element annotations. Specifically, we select three perspectives (legal fact, dispute focus, and law statutory) and build a comprehensive and structured label schema of legal elements for each of them, to enable accurate and knowledgeable evaluation of case similarities. The constructed dataset originates from Chinese civil cases and contains 100 query cases and 4,024 candidate cases. We implement several text classification algorithms for legal element prediction and various retrieval methods for retrieving similar cases on MUSER. The experimental results indicate that incorporating legal elements can benefit the performance of SCR models, but further efforts are still required to address the remaining challenges posed by MUSER. The source code and dataset are released at https://github.com/THUlawtech/MUSER. CCS CONCEPTS• Applied computing → Law.

show abstract

Investigating User Behavior in Legal Case Retrieval

Cited by 17 publications

References 36 publications

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

Legal Element-oriented Modeling with Multi-view Contrastive Learning for Legal Case Retrieval

MUSER: A Multi-View Similar Case Retrieval Dataset

Contact Info

Product

Resources

About