An Information Retrieval Pipeline for Legislative Documents from the Brazilian Chamber of Deputies

Abstract

This work investigates information retrieval methods to address the existing difficulties on the Preliminary Search, part of the law making process from the Brazilian Chamber of Deputies. For such, different preprocessing approaches, stemmers, language models, and BM25 variants were compared. Two legislative corpora from Chamber were used to build and validate the pipeline. All texts were converted to lowercase and had stopwords, accentuation, and punctuation removed. Words were represented by their stem combined with word unigram and bigram language models. Retrieving the bill that was originated from a specific job request, the BM25L with Savoy stemmer reached a R@20 of 0.7356. After removing queries with inconsistencies or which made reference exclusively to attachments, to other job requests, or to bills, the R@20 increased to 0.94.

Building a Relevance Feedback Corpus for Legal Information Retrieval in the Real-Case Scenario of the Brazilian Chamber of Deputies

“…Souza et al [11,28] investigated IR methods and presented a pipeline for the retrieval of legislative documents within the context of the Brazilian Chamber of Deputies. Evaluating the use of three variants of the BM25 algorithm, along with different pre-processing techniques, they built the IR model currently employed by the Chamber to retrieve bills and other queries relevant to a parliamentarian's request.…”

“…Nowadays, the IR model used by Conle to automatically retrieve relevant documents is based on BM25L [38] and a combination of pre-processing techniques: punctuation, accentuation, and stopwords removal + Stemming, with the Savoy algorithm [39], + unigram and bigram; as presented by [11]. BM25L ranks the documents by estimating their relevance to a query.…”

“…Actual consultations could not be used due to their confidentiality, as they contain private information about the parliamentarians. This made it impossible for the datasets used by [11,22,28] to be made available.…”

“…Besides constructing a corpus that can be utilized to evaluate other IR techniques, we initially employed this feedback data to measure the performance of the model used by Conle [11] in a real-case scenario. To perform this evaluation, we considered both somewhat relevant and very relevant documents as the correct set for each query.…”

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

“…Information Retrieval techniques are at the core of many legal research platforms, such as Lexis+ 2 and Westlaw Edge 3 , and have several critical application scenarios, such as legislative document retrieval [82], [83] and case retrieval [84], [85]. Depending on the application, the queries can be short (e.g., simple keywords), of medium length (e.g., Boolean or natural language queries) or long (e.g., whole documents).…”