Joint Learning of Word and Label Embeddings for Sequence Labelling in Spoken Language Understanding

Abstract

We propose an architecture to jointly learn word and label embeddings for slot filling in spoken language understanding. The proposed approach encodes labels using a combination of word embeddings and straightforward word-label association from the training data. Compared to the state-ofthe-art methods, our approach does not require label embeddings as part of the input and therefore lends itself nicely to a wide range of model architectures. In addition, our architecture computes contextual distances between words and labels to avoid adding contextual windows, thus reducing memory footprint. We validate the approach on established spoken dialogue datasets and show that it can achieve state-of-the-art performance with much fewer trainable parameters.

Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding

“…Chen et al [34] exploit label embedding for intent detection but not suitable for slot filling. Some works choose to extract label embeddings from data samples [38], [39], [40], [41] by exploiting words of values tagged with a semantic label, but they only focus on the corresponding values. We consider both the value and context information in the slot exemplar encoding.…”

Augmenting Slot Values and Contexts for Spoken Language Understanding with Pretrained Models

“…Since SLU is proved to exert significant influence on the final performance of dialogue systems [1], improving SLU performance is a crucial problem and attracts much attention in both academia and industry. Traditionally, SLU is trained in a supervised way with sufficient labeled data, achieving excellent performance [2,3,4]. Unfortunately, it is difficult and expensive to acquire enough labeled data in practice.…”