Feedback-Based Self-Learning in Large-Scale Conversational AI Agents

Abstract

Today, most of the large-scale conversational AI agents such as Alexa, Siri, or Google Assistant are built using manually annotated data to train the different components of the system including automatic speech recognition (ASR), natural language understanding (NLU), and entity resolution (ER). Typically, the accuracy of the machine learning models in these components are improved by manually transcribing and annotating data. As the scope of these systems increase to cover more scenarios and domains, manual annotation to improve the accuracy of these components becomes prohibitively costly and time con-suming. In this paper, we propose a system that leverages customer/system interaction feedback signals to automate learning without any manual annotation. Users of these systems tend to modify a previous query in hopes of fixing an error in the previous turn to get the right results. These reformulations, which are often preceded by defective experiences caused by either errors in ASR, NLU, ER, or the application. In some cases, users may not properly formulate their requests (e.g., providing partial title of a song), but gleaning across a wider pool of users and sessions reveals the underlying recurrent patterns. Our proposed self-learning system automatically detects the errors, generates reformulations, and deploys fixes to the runtime system to correct different types of errors occurring in different components of the system. In particular, we propose leveraging an absorbing Markov Chain model as a collaborative filtering mechanism in a novel attempt to mine these patterns, and coupling it with a guardrail rewrite selection mechanism that reactively evaluates these fixes using feedback friction data. We show that our approach is highly scalable, and able to learn reformulations that reduce Alexa-user errors by pooling anonymized data across millions of customers. The proposed self-learning system achieves a win-loss ratio of 11.8 and effectively reduces the defect rate by more than 30 percent on utterance level reformulations in our production A/B tests. To the best of our knowledge, this is the first self-learning large-scale conversational AI system in production.

An Overview of the Necessary Technologies and Implementation of an AI based Help Desk

“…It explains the principle of Rasa NLU, strategies of entity extraction like Rasa NLU technique and Neural Networks technique and analyses the system by analyzing intent recognition and entity extraction. It builds a bot that gives information about stocks; The general steps are summarized as following: (1) Send a message to the financial chatbot; (2) Sentences are analyzed and entities are returned by RASA NLU; (3) iex-finance API gives information about stocks; (4) Possible intent of the message chatbot, received by regular expression and keywords is returned; (5) Response to the message is given according to the intents and current states based on the state machine.…”

Mondegreen: A Post-Processing Solution to Speech Recognition Error Correction for Voice Search Queries

“…The most related line of work to Mondegreen is the work by Ponnusamy et al [19], in the context of Amazon's Alexa. Ponnusamy et al collect a dataset consisting of sequences of user interactions with Alexa.…”

Reinforcement Learning from Reformulations in Conversational Question Answering over Knowledge Graphs

“…In our work, we revive the more traditional sense of reformulations [12,15,27,30], where users pose queries in a different way when system responses are unsatisfactory. Several works on search and QA apply RL to automatically generate or retrieve reformulations that would proactively result in the best system response [10,16,43,46]. In contrast, Conqer learns from free-form user-generated reformulations.…”

“…𝑎𝑛𝑠 5 : Zendaya Coleman Utterances can be colloquial (𝑞 4 ) and incomplete (𝑞 2 , 𝑞 3 ), and inferring the proper context is a challenge (𝑞 5 ). Users can provide feedback in the form of question reformulations [46]: when an answer is incorrect, users may rephrase the question, hoping for better results. While users never know the correct answer upfront, they may often guess non-relevance when the answer does not match the expected type (director instead of movie) or from additional background knowledge.…”