Intent Discovery Through Unsupervised Semantic Text Clustering

Abstract

Conversational systems need to understand spoken language to be able to converse with a human in a meaningful coherent manner. This understanding (Spoken Language understanding-SLU) of the human language is operationalized through identifying intents and entities. While classification methods that rely on labeled data are often used for SLU, creating large supervised data sets is extremely tedious and time consuming. This paper presents a practical approach to automate the process of intent discovery on unlabeled data sets of human language text through clustering techniques. We explore a range of representations for the texts and various clustering methods to validate the clustering stability through quantitative metrics like Adjusted Random Index (ARI). A final alignment of the clusters to the semantic intent is determined through consensus labelling. Our experiments on public datasets demonstrate the effectiveness of our approach generating homogeneous clusters with 89% cluster accuracy, leading to better semantic intent alignments. Furthermore, we illustrate that the clustering offer an alternate and effective way to mine sentence variants that can aid the bootstrapping of SLU models.

On the Use of Transformer-Based Models for Intent Detection Using Clustering Algorithms

“…It also clusters the user dialog intents by employing hierarchical clustering and displays superior intent clustering results compared to conventional approaches that use K-Means. An ensemble approach was used in a different study to discover semantically related intents by comparing multiple classical word embedding methods, including Word2Vec and GloVe [12].…”

“…One of the challenges with clustering data is determining the appropriate number of partitions to use. A usual approach is to set the number of partitions to a value greater than the number of class labels determined by ground truth [12]. A possible way of addressing this problem is by using density-based clustering algorithms, for example, Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN), that are capable of retrieving dynamically formed natural clusters [13].…”

Bridge Pre-Training and Clustering: A Unified Contrastive Learning Framework for OOD Intent Discovery

“…We classify the existing methods of OOD discovery into two main categories, unsupervised and semi-supervised OOD discovery. Unsupervised methods [10][11][12] only model OOD data but ignore prior knowledge of indomain data thus impair final clustering performance. Therefore, recent work focus on the semi-supervised setting where there exist a few labeled IND intents [4,5].…”

Disentangled Knowledge Transfer for OOD Intent Discovery with Unified Contrastive Learning

“…Previous unsupervised OOD discovery models (Hakkani-Tür et al, 2015;Padmasundari and Bangalore, 2018;Shi et al, 2018) only model OOD data but ignore prior knowledge of in-domain data thus suffer from poor performance. Therefore, recent work (Lin et al, 2020; focus more on the semi-supervised setting where they firstly pre-train an in-domain intent classifier then perform clustering algorithms on extracted OOD intent representations by the pre-trained IND intent classifier.…”