Dataset Expansion with Pseudo-Labeling: Case Study for Optimizing Chatbot Intent Recognition

Abstract

Purpose of the study: This study presents an approach for improving the performance of natural language processing (NLP) models in pseudo-labeling tasks, with a particular focus on enhancing chatbot model intent recognition for business use cases. Methodology: The employed case study approach explores the pseudo-labeling technique and demonstrates a practical and efficient way to iteratively expand the original set of labeled data for the purpose of refining model training to achieve superior intent recognition accuracy in chatbots. Main Findings: The approach results in notable increases in macro-averaged F1 score and overall accuracy, particularly by iteratively re-training the model with progressively larger datasets. While enhancing the model's ability to generalize through difficult cases was effective, the study found that incorporating a full range of examples, including easy ones, yielded the best results. This comprehensive approach made the model better suited for real-world applications. Applications of the study: As chatbots are increasingly deployed in various sectors, including business, customer service, healthcare, and education, it becomes crucial for research to examine their long-term impact, scalability, and adaptability to ensure their effectiveness and sustainability in diverse contexts. Therefore, building more accurate chatbots, capable of understanding a wide range of user intents, is particularly valuable in real-world applications where chatbots need to respond to diverse, often complex and unpredictable user queries. Novelty/Originality of the study: Unlike traditional approaches, this study introduces a novel strategy of filling low-density regions in the dataset with pseudo-labels, allowing the model to better separate classes and handle semantically similar but varied messages. These advancements contribute to a more effective and scalable chatbot solution across diverse industries.