WhenTheFact: Extracting Events from European Legal Decisions

Navas-Loro, María; Rodrı́guez-Doncel, Victor

doi:10.3233/faia220470

Cited by 1 publication

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous works involving ECtHR corpus has dealt with COC (Aletras et al, 2016;Chalkidis et al, 2019;Valvoda et al, 2023;Santosh et al, 2022Santosh et al, , 2023Tyss et al, 2023;, argu-ment mining (Mochales and Moens, 2008;Habernal et al, 2023;Poudyal et al, 2019Poudyal et al, , 2020, event extraction (Filtz et al, 2020;Navas-Loro and Rodriguez-Doncel, 2022) and vulnerability type classification (Xu et al, 2023). In this work, we focus mainly on COC task using ECtHR corpus studying the sources of disagreement/HLV in COC rationale, to contribute to the field of explainable legal NLP.…”

Section: Tasks On Ecthr Corporamentioning

confidence: 99%

From Dissonance to Insights: Dissecting Disagreements in Rationale Construction for Case Outcome Classification

Xu,

T.y.s.s,

Ichim

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

In legal NLP, Case Outcome Classification (COC) must not only be accurate but also trustworthy and explainable. Existing work in explainable COC has been limited to annotations by a single expert. However, it is well-known that lawyers may disagree in their assessment of case facts. We hence collect a novel dataset RAVE: Rationale Variation in ECHR 1 , which is obtained from two experts in the domain of international human rights law, for whom we observe weak agreement. We study their disagreements and build a two-level task-independent taxonomy, supplemented with COC-specific subcategories. We quantitatively assess different taxonomy categories and find that disagreements mainly stem from underspecification of the legal context, which poses challenges given the typically limited granularity and noise in COC metadata. To our knowledge, this is the first work in the legal NLP that focuses on building a taxonomy over human label variation. We further assess the explainablility of state-of-the-art COC models on RAVE and observe limited agreement between models and experts. Overall, our case study reveals hitherto underappreciated complexities in creating benchmark datasets in legal NLP that revolve around identifying aspects of a case's facts supposedly relevant to its outcome.

show abstract