It is increasingly common to use chatbots as an interface to services. Making this experience more humanized requires the chatbot to understand natural language and express itself using it. One of the main components of a chatbot is Natural Language Understanding (NLU) model, which is responsible for interpreting the text and extracting the intent and entities present in that text. It's possible to focus only on one of these tasks of NLU, such as Intent classification. To train an NLU intent classification model, it's generally necessary to use a considerable amount of annotated data, where each sentence of the dataset receives a label indicating an intent.Performing manually labeling data is an arduous and impracticable process depending on the data volume. Thus, an unsupervised machine learning technique, such as data clustering, could be applied to find patterns in the data and label them. For this task, it is essential to have an effective vector embedding representation of texts that depicts the semantic information and helps the machine understand the context, intent, and other nuances of the entire text. In this paper, we perform an extensive evaluation of different text embedding models for clustering and labeling, we also apply some operations to improve the quality of the dataset, such as removing sentences establishing different strategies for distance thresholds in terms of (cosine similarity) for the clusters' centroids. Then we trained some Intent Classification Models with two different architectures, one built with the Rasa framework and the other with a Neural Network (NN) using the text of attendance from the Coronavirus Platform Service of Ceará, Brazil. We also manually annotated a dataset to be used as validation data. We found that semi-automatic labeling through clustering and visual inspection added some biases to the intent classification models. However, we still achieved competitive results in terms of accuracy for the trained models.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.