Structured Prediction Networks through Latent Cost Learning

Milidiú, Ruy Luiz; Rocha, Rafael Henrique Santos

doi:10.1109/ssci.2018.8628625

Cited by 2 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While in optimization problems local solutions often produce optimal results, structured prediction represents a valid alternative to solve NLP tasks requiring complex output, such as syntactic parsing (Roth and Yih, 2004), co-reference resolution (Yu and Joachims, 2009;Fernan-des et al, 2014), and clustering (Finley and Joachims, 2005;Haponchyk et al, 2018). Nonetheless, relatively few works extend structured prediction theory to deep learning Durrett and Klein, 2015;Weiss et al, 2015;Kiperwasser and Goldberg, 2016;Peng et al, 2018;Milidiú and Rocha, 2018;Wang et al, 2019). In particular, when it comes to clustering, designing a differentiable loss function that captures the global characteristics of good clustering is particularly hard; for this reason, when dealing with coreference resolution -a closely related task - Lee et al (2017) use simple losses, which already perform well but do not strictly take into account the cluster structure.…”

Section: Structured Predictionmentioning

confidence: 99%

Supervised Clustering Loss for Clustering-Friendly Sentence Embeddings: an Application to Intent Clustering

Barnabò,

Uva,

Pollastrini

et al. 2023

Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings)

View full text Add to dashboard Cite

Modern virtual assistants are trained to classify customer requests into a taxonomy of predesigned intents. Requests that fall outside of this taxonomy, however, are often unhandled and need to be clustered to define new experiences. Recently, state-of-the-art results in intent clustering were achieved by training a neural network with a latent structured prediction loss. Unfortunately, though, this new approach suffers from a quadratic bottleneck as it requires to compute a joint embedding representation for all pairs of utterances to cluster. To overcome this limitation, we instead cast the problem into a representation learning task, and we adapt the latent structured prediction loss to fine-tune sentence encoders, thus making it possible to obtain clustering-friendly single-sentence embeddings. Our experiments show that the supervised clustering loss returns state-of-the-art results in terms of clustering accuracy and adjusted mutual information.

show abstract

Section: Structured Predictionmentioning

confidence: 99%

Supervised Clustering Loss for Clustering-Friendly Sentence Embeddings: an Application to Intent Clustering

Barnabò,

Uva,

Pollastrini

et al. 2023

Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings)

View full text Add to dashboard Cite

show abstract

Supervised Neural Clustering via Latent Structured Output Learning: Application to Question Intents

Haponchyk

Moschitti

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Previous pre-neural work on structured prediction has produced very effective supervised clustering algorithms using linear classifiers, e.g., structured SVM or perceptron. However, these cannot exploit the representation learning ability of neural networks, which would make supervised clustering even more powerful, i.e., general clustering patterns can be learned automatically. In this paper, we design neural networks based on latent structured prediction loss and Transformer models to approach supervised clustering. We tested our methods on the task of automatically recreating categories of intents from publicly available question intent corpora. The results show that our approach delivers 95.65% of F1, outperforming the state of the art by 17.24%.

show abstract

Structured Prediction Networks through Latent Cost Learning

Cited by 2 publications

References 8 publications

Supervised Clustering Loss for Clustering-Friendly Sentence Embeddings: an Application to Intent Clustering

Supervised Clustering Loss for Clustering-Friendly Sentence Embeddings: an Application to Intent Clustering

Supervised Neural Clustering via Latent Structured Output Learning: Application to Question Intents

Contact Info

Product

Resources

About