Cross-lingual Annotation Projection in Legal Texts

Abstract

We study annotation projection in text classification problems where source documents are published in multiple languages and may not be an exact translation of one another. In particular, we focus on the detection of unfair clauses in privacy policies and terms of service. We present the first English-German parallel asymmetric corpus for the task at hand. We study and compare several language-agnostic sentence-level projection methods. Our results indicate that a combination of word embeddings and dynamic time warping performs best.

A Corpus for Multilingual Analysis of Online Terms of Service

“…We employ the same methodology proposed in Galassi et al (2020), by extending the study to two additional languages, namely Italian and Polish. The general idea of the methodology is to transfer the annotations of a given English document onto the same document, given in a different language.…”

“…From a machine learning point of view, the question is whether we shall necessarily train independent models for each and every language. This paper only partially answers the question, by extending previous work (Galassi et al, 2020) towards three different languages and by making a novel multilingual corpus available to the community. Future research will consider training and comparing the classifiers.…”

MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer

“…They experimented with monolingual SVM classifiers and their combination as a multilingual ensemble. More recently, Galassi et al (2020) transferred sentence-level gold labels from annotated English to non-annotated German sentences, for the task of identifying unfair clauses in Terms of Service (2.7k sentences) and Privacy Policy documents (1.8k). They experimented with similarity-based methods aligning the English sentences to machine-translations of the German sentences.…”

An Argumentative Dialogue System for COVID-19 Vaccine Information

“…The similarity between two embeddings can be computed using any similarity function that operates on high-dimensional numerical vectors. We use the Bray-Curtis similarity [3] since it has led to satisfactory results in previous works [17], but other measures, such as cosine similarity [20], may be valid alternative. A possible alternative to the use of sentence embeddings combined with a similarity measure may be the use of neural architectures specifically trained to perform this task, such as cross-encoders [29].…”