Unsupervised law article mining based on deep pre-trained language representation models with application to the Italian civil code

Abstract

Modeling law search and retrieval as prediction problems has recently emerged as a predominant approach in law intelligence. Focusing on the law article retrieval task, we present a deep learning framework named LamBERTa, which is designed for civil-law codes, and specifically trained on the Italian civil code. To our knowledge, this is the first study proposing an advanced approach to law article prediction for the Italian legal system based on a BERT (Bidirectional Encoder Representations from Transformers) learning framework, which has recently attracted increased attention among deep learning approaches, showing outstanding effectiveness in several natural language processing and learning tasks. We define LamBERTa models by fine-tuning an Italian pre-trained BERT on the Italian civil code or its portions, for law article retrieval as a classification task. One key aspect of our LamBERTa framework is that we conceived it to address an extreme classification scenario, which is characterized by a high number of classes, the few-shot learning problem, and the lack of test query benchmarks for Italian legal prediction tasks. To solve such issues, we define different methods for the unsupervised labeling of the law articles, which can in principle be applied to any law article code system. We provide insights into the explainability and interpretability of our LamBERTa models, and we present an extensive experimental analysis over query sets of different type, for single-label as well as multi-label evaluation tasks. Empirical evidence has shown the effectiveness of LamBERTa, and also its superiority against widely used deep-learning text classifiers and a few-shot learner conceived for an attribute-aware prediction task.

Predicting the number of days in court cases using artificial intelligence

“…In [ 14 ], an approach for classifying articles of law for the Italian legal system based on a learning framework based on Bidirectional Encoder Representations from Transformers (BERT) was presented. Other examples are presented in [ 15 ], which performed experiments to detect through the analysis of judgments of the European Court of Human Rights whether the case was judged as a violation of someone’s right or not, and [ 16 ] used Glove word vectors generated for the Portuguese Language and Convolutional Neural Network (CNN) to perform the classification of types of documents in judicial proceedings of the Court of Justice of Minas Gerais (TJMG).…”

Thirty years of Artificial Intelligence and Law: the second decade

“…Clearly, advances in NLP are needed. Research on this is already being done, (Branting et al 2021), 26 including the first legal applications of currently fashionable language models like BERT (Chalkidis et al 2020;Zheng et al 2021b;Tagarelli and Simeri 2021). Moreover, given the arguably limited generality of the symbolic legal-reasoning models underlying SMILE-IBP and VJAP, further work on developing such models is also needed.…”

Automatic explanation of the classification of Spanish legal judgments in jurisdiction-dependent law categories with tree estimators

“…Moreover, more recent studies follow semi-supervised techniques to extract relevant concepts from judicial texts (Thomas & Sangeetha, 2021). Other works apply Naive Bayes and other supervised methods (Medvedeva et al, 2020;Hausladen et al, 2020) and transformer-based language models and other Deep Neural Network (dnn) architectures, such as that by Tagarelli & Simeri (2021), a fine-tuned version of Italian bert 1 (Devlin et al, 2019) for law article classification; and the Convolutional Neural Network (cnn) for multi-label Chinese legal text classification by Qiu et al (2020). Unfortunately, most pre-trained language models' embeddings are not adapted to the legal domain (Beltagy et al, 2019), so they tend to underperform compared to traditional ml algorithms like rf, as noted by Chen et al (2022).…”