A Sentence-Level Hierarchical BERT Model for Document Classification with Limited Labelled Data

Lu, Jinghui; Henchion, Maeve; Bacher, Ivan; Namee, Brian Mac

doi:10.1007/978-3-030-88942-5_18

Cited by 14 publications

(10 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach is to leverage domain knowledge in conjunction with a state-of-the-art H-BERT architecture [24]. We use a Python implementation of the ADF constructed specifically for Article 6 of the ECHR [18] to provide intermediate classifications of base-level factors, and train an independent H-BERT model for each base-level factor.…”

Section: Hybrid Adf/h-bert Methodsmentioning

confidence: 99%

“…The scarcity of the data is in stark contrast to the relatively vast data sets that are usually employed for NLP tasks. We focus on two classification approaches, a state-ofthe-art hierarchical BERT approach developed specifically for small data sets which we refer to as H-BERT [24], and our hybrid system which uses the aforementioned H-BERT architecture in conjunction with the ADF layer as outlined in section 4. Both approaches use the same fact-level pre-trained RoBERTa model encodings using 512 tokens, and both use 256 tokens for document BERT model encoding.…”

Section: Data Set and Implementation Detailsmentioning

confidence: 99%

See 1 more Smart Citation

Reasoning with Legal Cases: A Hybrid ADF-ML Approach

Mumford

Atkinson

Bench‐Capon

2022

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Reasoning with legal cases has long been modelled using symbolic methods. In recent years, the increased availability of legal data together with improved machine learning techniques has led to an explosion of interest in data-driven methods being applied to the problem of predicting outcomes of legal cases. Although encouraging results have been reported, they are unable to justify the outcomes produced in satisfactory legal terms and do not exploit the structure inherent within legal domains; in particular, with respect to the issues and factors relevant to the decision. In this paper we present the technical foundations of a novel hybrid approach to reasoning with legal cases, using Abstract Dialectical Frameworks (ADFs) in conjunction with hierarchical BERT. ADFs are used to represent the legal knowledge of a domain in a structured way to enable justifications and improve performance. The machine learning is targeted at the task of factor ascription; once factors present in a case are ascribed, the outcome follows from reasoning over the ADF. To realise this hybrid approach, we present a new hybrid system to enable factor ascription, envisioned for use in legal domains, such as the European Convention on Human Rights that is used frequently in modelling experiments.

show abstract

Section: Hybrid Adf/h-bert Methodsmentioning

confidence: 99%

Section: Data Set and Implementation Detailsmentioning

confidence: 99%

Reasoning with Legal Cases: A Hybrid ADF-ML Approach

Mumford

Atkinson

Bench‐Capon

2022

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

show abstract

“…The reason for using JSD divergence instead of KL divergence is that there may be significant differences between the current policy and the past policy, making the calculation of KL divergence difficult or even impossible. JSD divergence effectively alleviates this problem [27]. If all the oversamples in the sampling batch match the distribution under the current policy, then ρ = 0; when the oversamples match the distribution under the current policy to some extent, then ρ ∈ (0, ∞ ).…”

Section: Off‐policy Correction Algorithmmentioning

confidence: 99%

Off‐policy correction algorithm for double Q network based on deep reinforcement learning

Zhang,

Liu,

Wang

et al. 2023

IET Cyber-Syst and Robotics

View full text Add to dashboard Cite

A deep reinforcement learning (DRL) method based on the deep deterministic policy gradient (DDPG) algorithm is proposed to address the problems of a mismatch between the needed training samples and the actual training samples during the training of intelligence, the overestimation and underestimation of the existence of Q‐values, and the insufficient dynamism of the intelligence policy exploration. This method introduces the Actor‐Critic Off‐Policy Correction (AC‐Off‐POC) reinforcement learning framework and an improved double Q‐value learning method, which enables the value function network in the target task to provide a more accurate evaluation of the policy network and converge to the optimal policy more quickly and stably to obtain higher value returns. The method is applied to multiple MuJoCo tasks on the Open AI Gym simulation platform. The experimental results show that it is better than the DDPG algorithm based solely on the different policy correction framework (AC‐Off‐POC) and the conventional DRL algorithm. The value of returns and stability of the double‐Q‐network off‐policy correction algorithm for the deep deterministic policy gradient (DCAOP‐DDPG) proposed by the authors are significantly higher than those of other DRL algorithms.

show abstract

“…For instance, Kulesza et al (2010) suggests that when a model is trained with a small subset of labelled data, it is prone to exploiting spurious patterns leading to poor generalisability that is evident in the performance decay in outof-distribution (OOD) datasets. In spite of these issues, training deep neural networks using few labelled examples is a compelling scenario since unlabelled data may be abundant but labelled data is expensive to obtain in real-world applications (Lu and MacNamee, 2020;Lu et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

A Rationale-Centric Framework for Human-in-the-loop Machine Learning

Liang¹,

Yang²,

Namee³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We present a novel rationale-centric framework with human-in-the-loop -Rationalescentric Double-robustness Learning (RDL) -to boost model out-of-distribution performance in few-shot learning scenarios. By using static semi-factual generation and dynamic human-intervened correction, RDL exploits rationales (i.e. phrases that cause the prediction), human interventions and semi-factual augmentations to decouple spurious associations and bias models towards generally applicable underlying distributions, which enables fast and accurate generalisation. Experimental results show that RDL leads to significant prediction benefits on both in-distribution and out-of-distribution tests compared to many state-of-the-art benchmarks-especially for few-shot learning scenarios. We also perform extensive ablation studies to support in-depth analyses of each component in our framework.

show abstract

A Sentence-Level Hierarchical BERT Model for Document Classification with Limited Labelled Data

Cited by 14 publications

References 17 publications

Reasoning with Legal Cases: A Hybrid ADF-ML Approach

Reasoning with Legal Cases: A Hybrid ADF-ML Approach

Off‐policy correction algorithm for double Q network based on deep reinforcement learning

A Rationale-Centric Framework for Human-in-the-loop Machine Learning

Contact Info

Product

Resources

About