Open Intent Discovery through Unsupervised Semantic Clustering and Dependency Parsing

Abstract

Intent understanding plays an important role in dialog systems, and is typically formulated as a supervised classification problem. However, it is challenging and time-consuming to design the intent labels manually to support a new domain. This paper proposes an unsupervised two-stage approach to discover intents and generate meaningful intent labels automatically from a collection of unlabeled utterances. In the first stage, we aim to generate a set of semantically coherent clusters where the utterances within each cluster convey the same intent. We obtain the utterance representation from various pre-trained sentence embeddings and present a metric of balanced score to determine the optimal number of clusters in K-means clustering. In the second stage, the objective is to generate an intent label automatically for each cluster. We extract the ACTION-OBJECT pair from each utterance using a dependency parser and take the most frequent pair within each cluster, e.g., book-restaurant, as the generated cluster label. We empirically show that the proposed unsupervised approach can generate meaningful intent labels automatically and achieves high precision and recall in utterance clustering and intent discovery.

Z-BERT-A: a zero-shot Pipeline for Unknown Intent detection

“…The main limitation of Z-BERT-A currently lies in the new intent generation stage that is relying extensively on the quality of the dependency parsing. An interesting avenue to explore in the future consists in relying on zero-shot learning approaches even in the intent generation phase (Liu et al 2021) without compromising in terms of model size and inference requirements. Z-BERT-A is available at the following link: https://github.com/GT4SD/zberta.…”

“…Liu et al (2022) proposed an approach leveraging Transformers-based architectures, while the majority relied on RNN architectures like LSTMs (Xia et al 2018). For the problem we are focusing here, i.e., finding novel intents, there are have been two interesting attempts to propose a pipeline for generation and extraction (Vedula et al 2019;Liu et al 2021). Liu et al (2021) addressed the new intent discovery as a clustering problem, proposing an adaptation of K-means clustering.…”

“…For the problem we are focusing here, i.e., finding novel intents, there are have been two interesting attempts to propose a pipeline for generation and extraction (Vedula et al 2019;Liu et al 2021). Liu et al (2021) addressed the new intent discovery as a clustering problem, proposing an adaptation of K-means clustering. Here, dependency parsing is used to extract from each cluster a mean ACTION-OBJECT pair representing the common emerging intent for that particular cluster.…”

“…To generate the set of potential novel intents, we extract from an input sentence through the dependency parser a pair of words obtained by searching for specific Arc-Relations (AR) in the dependency tree of the parsed sentence. Since an intent is usually composed by an action-object pair (Vedula et al 2019;Liu et al 2021), we exploit this pattern in order to search for DOBJ, compound and AMOD arc relations. We perform a four level detection, which means finding the four main relations which can generate a base-intent.…”

Zero-Shot-BERT-Adapters: a Zero-Shot Pipeline for Unknown Intent Detection

“…(Liu et al, 2022) proposed an approach leveraging Transformers-based architectures, while most relied on RNN architectures like LSTMs (Xia et al, 2018). For our problem of interest, two notable efforts have attempted to address the issue of unseen intent detection (Vedula et al, 2019;Liu et al, 2021). (Liu et al, 2021) handled novel intent discovery as a clustering problem, proposing an adaptation of K-means.…”