Learning to Ask for Conversational Machine Learning

Abstract

Natural language has recently been increasingly explored as a medium of supervision for training machine learning models. Here, we explore learning classification tasks using language in a conversational setting-where the automated learner does not simply receive language input from a teacher, but can proactively engage the teacher by asking template-based questions. We experiment with a reinforcement learning framework, where the learner's actions correspond to question types and the reward for asking a question is based on how the teacher's response changes performance of the resulting machine learning model on the learning task. In this framework, learning good question-asking strategies corresponds to asking sequences of questions that maximize the cumulative (discounted) reward, and hence quickly lead to effective classifiers. Empirical analysis shows that learned question-asking strategies can expedite classifier training by asking appropriate questions at different points in the learning process. The approach allows learning using a blend of strategies, including learning from observations, explanations and clarifications.