FHTC: Few-Shot Hierarchical Text Classification in Financial Domain

Wang, Anqi; Chen, Qingcai; Li, Dongfang

doi:10.1007/978-3-030-92270-2_56

Cited by 2 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gradients can capture intrinsic task-related characteristics, playing a strong predictive role in model optimization on a dataset, as evidenced by recent studies 43 – 46 . Grad2Task 43 posits that gradients can be used as features to capture task nature and distinguish between tasks under a meta-learning framework. Numerous gradient surgery techniques 44 , 45 have been proposed to address gradient conflict in multitask learning.…”

Section: Resultsmentioning

confidence: 93%

Fast and effective molecular property prediction with transferability map

Yao,

Song,

Jia

et al. 2024

Commun Chem

View full text Add to dashboard Cite

Effective transfer learning for molecular property prediction has shown considerable strength in addressing insufficient labeled molecules. Many existing methods either disregard the quantitative relationship between source and target properties, risking negative transfer, or require intensive training on target tasks. To quantify transferability concerning task-relatedness, we propose Principal Gradient-based Measurement (PGM) for transferring molecular property prediction ability. First, we design an optimization-free scheme to calculate a principal gradient for approximating the direction of model optimization on a molecular property prediction dataset. We have analyzed the close connection between the principal gradient and model optimization through mathematical proof. PGM measures the transferability as the distance between the principal gradient obtained from the source dataset and that derived from the target dataset. Then, we perform PGM on various molecular property prediction datasets to build a quantitative transferability map for source dataset selection. Finally, we evaluate PGM on multiple combinations of transfer learning tasks across 12 benchmark molecular property prediction datasets and demonstrate that it can serve as fast and effective guidance to improve the performance of a target task. This work contributes to more efficient discovery of drugs, materials, and catalysts by offering a task-relatedness quantification prior to transfer learning and understanding the relationship between chemical properties.

show abstract

Section: Resultsmentioning

confidence: 93%

Fast and effective molecular property prediction with transferability map

Yao,

Song,

Jia

et al. 2024

Commun Chem

View full text Add to dashboard Cite

show abstract

“…Due to the cost of acquiring high-quality labeled data, researchers have been working on improving model performances under low-resource settings. A special case is fewshot learning (Wang et al 2021;Tsimpoukelli et al 2021;Perez, Kiela, and Cho 2021) in which data is severely limited. Earlier works address data sparsity through large unlabeled datasets (Gunel et al 2020;Artetxe et al 2018), data augmentation (Salamon and Bello 2017;Fadaee, Bisazza, and Monz 2017), or the removal of redundant information in the pretraining representation (Belinkov, Henderson et al 2021).…”

Section: Low-resourcementioning

confidence: 99%

Prototypical Fine-Tuning: Towards Robust Performance under Varying Data Sizes

Jin

Wang

Hao

et al. 2023

AAAI

View full text Add to dashboard Cite

In this paper, we move towards combining large parametric models with non-parametric prototypical networks. We propose prototypical fine-tuning, a novel prototypical framework for fine-tuning pretrained language models (LM), which automatically learns a bias to improve predictive performance for varying data sizes, especially low-resource settings. Our prototypical fine-tuning approach can automatically adjust the model capacity according to the number of data points and the model's inherent attributes. Moreover, we propose four principles for effective prototype fine-tuning towards the optimal solution. Experimental results across various datasets show that our work achieves significant performance improvements under various low-resource settings, as well as comparable and usually better performances in high-resource scenarios.

show abstract

FHTC: Few-Shot Hierarchical Text Classification in Financial Domain

Cited by 2 publications

References 16 publications

Fast and effective molecular property prediction with transferability map

Fast and effective molecular property prediction with transferability map

Prototypical Fine-Tuning: Towards Robust Performance under Varying Data Sizes

Contact Info

Product

Resources

About