A scalable approach to legal question answering

Bennett, Zachary; Russell-Rose, Tony; Farmer, Kate

doi:10.1145/3086512.3086547

Cited by 9 publications

(9 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It provides a natural language interface to over 22 million single sentence summaries of case law, and high-confidence candidate answers are returned along with ad-hoc results returned from Westlaw's search system. Similarly, Bennett et al [8] demonstrated a system employed at LexisNexis that automatically identifies parts of documents that are extracted and treated as answers to be presented to users. This system suggests potential questions to users in a query box via autocompletion and returns an answer card above ad-hoc search results.…”

Section: Question Answeringmentioning

confidence: 99%

Case law retrieval: problems, methods, challenges and evaluations in the last 20 years

Locke¹,

Zuccon²

2022

Preprint

View full text Add to dashboard Cite

Case law retrieval is the retrieval of judicial decisions relevant to a legal question. Case law retrieval comprises a significant amount of a lawyer's time, and is important to ensure accurate advice and reduce workload. We survey methods for case law retrieval from the past 20 years and outline the problems and challenges facing evaluation of case law retrieval systems going forward. Limited published work has focused on improving ranking in ad-hoc case law retrieval. But there has been significant work in other areas of case law retrieval, and legal information retrieval generally. This is likely due to legal search providers being unwilling to give up the secrets of their success to competitors. Most evaluations of case law retrieval have been undertaken on small collections and focus on related tasks such as question-answer systems or recommender systems. Work has not focused on Cranfield style evaluations and baselines of methods for case law retrieval on publicly available test collections are not present. This presents a major challenge going forward. But there are reasons to question the extent of this problem, at least in a commercial setting. Without test collections to baseline approaches it cannot be known whether methods are promising. Works by commercial legal search providers show the effectiveness of natural language systems as well as query expansion for case law retrieval. Machine learning is being applied to more and more legal search tasks, and undoubtedly this represents the future of case law retrieval.

show abstract

Section: Question Answeringmentioning

confidence: 99%

Case law retrieval: problems, methods, challenges and evaluations in the last 20 years

Locke¹,

Zuccon²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Lex Machina, 6 now part of LexisNexis, predicts outcomes of new cases based on information about litigation participants and their behavior gleaned from a large repository of past cases (Surdeanu et al 2011). Another program uses machine learning to predict the outcomes of decisions of the European Court of Human Rights based on the cases' textual descriptions of case facts (Medvedeva 2020).…”

Section: New Legal Apps Change Legal Practicementioning

confidence: 99%

“…Commercial IR providers, of course, are constantly enhancing their own systems for cognitive computing. Students read a paper describing Lexis Answers, a recent addition to the Lexis Advance legal research platform (Bennett et al 2017). The paper presents the challenges that the system developers are addressing, for example, the need to identify irrelevant text, such as text quoted in the documents, the need to disambiguate terms by context, for example, distinguishing meanings of "fraud" in criminal and civil law, and the problem of creating an ontology and type system to support text understanding of Lexis users' queries.…”

Section: Applying ML To Legal Decisionsmentioning

confidence: 99%

A Law School Course in Applied Legal Analytics and AI

Šavelka¹,

Grabmair

Ashley

2021

LiC

View full text Add to dashboard Cite

Technological advances in artificial intelligence (AI) are affecting the legal profession. Machine learning (ML) and natural language processing (NLP) enable new legal apps that, to some extent, can analyze contracts, answer legal questions, or predict the outcome of a case or issue. While it is hard to predict the extent to which these techniques will change law practice, two things are certain: legal professionals will need to understand the new text analysis techniques and how to use and evaluate them, and law faculties face the question of how to teach law students the required skills and knowledge to do so. At the University of Pittsburgh School of Law, the authors have co-designed a semester-long course entitled, Applied Legal Data Analytics and AI, and twice taught it to combined groups of law students and students from technical departments. The course provides a hands-on practical introduction to applying ML and NLP to extract information from legal text data, the ways text analytics have been applied to support the work of legal professionals, researchers, and administrators, and the techniques for evaluating how well they work. The article introduces the new text analytic techniques and briefly surveys law schools’ current efforts to incorporate instruction on computer programming and machine learning in legal education. Then it describes the 2020 version of the course, including the students, instructors, and course sessions in overview. We explain how we taught law students skills of programming and experimental design and engaged them in assignments that involve using Python programming environments to analyze legal data. The course culminated in joint projects engaging small teams of law and technical students in applying machine learning and data analytics to legal problems. The article explains how the instructors prepare the students for the final course projects, beginning early in the term with project ideas and databases of text, forming teams, working on the projects as a team and obtaining interim feedback, and finally completing the projects and reporting results. We draw some salient comparisons between the 2019 and 2020 versions of the course and report what worked well and what did not, the students’ reactions, and lessons learned for future offerings of the course.

show abstract

“…Perhaps the most illustrative of these limitations is the inability of the systems to understand the question instead of For this reason, the existing approaches are focused on developing good ideas that allow to answer the questions asked and obviate the performance details since these can be solved with these new forms of high-performance computing. Some outstanding works in this direction are proposed by Bennet et al with a focus on the scalability of the solution by design (Bennett et al, 2017), or Mimouni et al for handling the problem of complex queries by working with approximate answers and richer semantics (Mimouni et al, 2017). Last but not least, there is also a novel approach based on the concept of mutual information exchange (Martinez-Gil et al, 2019a), which, when applied to large volumes of data, have demonstrated better performance than classic co-occurrence methods.…”

Section: Big Data Solutionsmentioning

confidence: 99%

A Survey on Legal Question Answering Systems

Martinez‐Gil¹

2021

Preprint

View full text Add to dashboard Cite

Many legal professionals think that the explosion of information about local, regional, national, and international legislation makes their practice more costly, time-consuming, and even error-prone. The two main reasons for this are that most legislation is usually unstructured, and the tremendous amount and pace with which laws are released causes information overload in their daily tasks. In the case of the legal domain, the research community agrees that a system allowing to generate automatic responses to legal questions could substantially impact many practical implications in daily activities.The degree of usefulness is such that even a semi-automatic solution could significantly help to reduce the workload to be faced. This is mainly because a Question Answering system could be able to automatically process a massive amount of legal resources to answer a question or doubt in seconds, which means that it could save resources in the form of effort, money, and time to many professionals in the legal sector. In this work, we quantitatively and qualitatively survey the solutions that currently exist to meet this challenge.

show abstract

A scalable approach to legal question answering

Abstract: Lexis Answers is a question answering service deployed within a live production system. In this paper we provide an overview of the system, an insight into some of the key AI challenges, and a brief description of current evaluation techniques.

Cited by 9 publications

References 3 publications

Case law retrieval: problems, methods, challenges and evaluations in the last 20 years

Case law retrieval: problems, methods, challenges and evaluations in the last 20 years

A Law School Course in Applied Legal Analytics and AI

A Survey on Legal Question Answering Systems

Contact Info

Product

Resources

About