Structured Refinement for Sequential Labeling

Abstract

Filtering target-irrelevant information through hierarchically refining hidden states has been demonstrated to be effective for obtaining informative representations. However, previous work simply relies on locally normalized attention without considering possible labels at other time steps, the capacity for modeling long-term dependency relations is thus limited. In this paper, we propose to extend previous work with globally normalized attention, e.g., structured attention, to leverage structural information for more effective representation refinement. We also propose two implementation tricks to accelerate CRF computation and an initialization trick for Chinese character embeddings to further improve performance. We provide extensive experimental results on various datasets to show the effectiveness and efficiency of our proposed method.

Joint Modelling of Slot Filling and Intent Detection in Constrained Resource Scenarios

“…Parallelly, the challenges posed by long-tail scenarios in sequence labelling (SL) tasks have been tackled by Wang et al [7] They proposed the Graph Neural Network for Sequence Labelling (GNN-SL), a technique that employs Graph Neural Networks (GNNs) to enhance SL model outputs by retrieving similar tag instances from the training dataset. The use of GNNs in constructing a heterogeneous graph allows for the propagation of information between retrieved tag samples and input word sequences.…”