A Taxonomy of Interactive Online Machine Learning Strategies

Abstract

In interactive machine learning, human users and learning algorithms work together in order to solve challenging learning problems, e.g. with limited or no annotated data or trust issues. As annotating data can be costly, it is important to minimize the amount of annotated data needed for training while still getting a high classification accuracy. This is done by attempting to select the most informative data instances for training, where the amount of instances is limited by a labelling budget. In an online learning setting, the decision of whether or not to select an instance for labelling has to be done on-the-fly, as the data arrives in a sequential order and is only valid for a limited time period. We present a taxonomy of interactive online machine learning strategies. An interactive learning strategy determines which instances to label in an unlabelled dataset. In the taxonomy we differentiate between interactive learning strategies when the computer controls the learning process (active learning) and those when human users control the learning process (machine teaching). We then make a distinction between what triggers the learning: active learning could be triggered by uncertainty, time, or randomly, whereas machine teaching could be triggered by errors, state changes, time, or factors related to the user. We also illustrate the taxonomy by implementing versions of the different strategies and performing experiments on a benchmark dataset as well as on a synthetically generated dataset. The results show that the choice of interactive learning strategy affects performance, especially in the beginning of the online learning process, when there is a limited amount of labelled data.

Proceedings of the Second Workshop on Bridging Human--Computer Interaction and Natural Language Processing

“…Olsson (2009) discussed AL for NLP tasks, while Schröder and Niekler (2020) discussed deep learning with AL. Our study also builds on Tegen et al (2020)'s use of simulation to study AL query strategies and MT assessment and teaching criteria. Lu and MacNamee (2020) reported on experiments where transformer-based representations performed consistently better than other text representations, taking advantage of the label information that arises in AL.…”

“…MT has been explored mostly in computing security, where the teacher is a hacker/advisor who selects training data to adjust the behavior of an adaptive, evolving learner (Alfeld et al, 2016(Alfeld et al, , 2017. Tegen et al (2020) reported that MT could greatly reduce the number of instances required, and even outperformed most AL strategies. These findings are compelling and motivate exploring MT's potential in NLP, which, however, has some distinct characteristics, including high-dimensional data impacted by scarcity.…”

“…for which it has the least probability of prediction in its most probable class; in margin sampling, instances with the smallest difference between its toptwo most likely classes; and in entropy, with the largest entropy (Olsson, 2009;Tegen et al, 2020).…”

“…In our study, the stakeholders in the learning process are models (learners) and humans (agent-users or agentteachers). Tegen et al (2020) posit that iML involves either Active Learning (AL) or interactive Machine Teaching (MT), 1 based on humans' role in the learning loop. In AL, the learning algorithm uses query strategies (e.g., triggered by uncertainty) to iteratively select instances from which it learns (Settles, 2009) if licensed by a budget; with a human agent who annotates upon learner request.…”

“…Interactive MT assumes human agent interaction with the learner (Liu et al, 2017), for enabling time-and resource-efficient model convergence. Following the training by error criterion described in Tegen et al (2020), if the learner is unable to predict the right answer, and the budget allows, the human teacher instructs the learner with the label. Thus, AL leverages measures to wisely choose instances for human labeling and subsequent learning, whereas MT capitalizes on the teacher's knowledge to wisely select training instances and proceed to learn when the criterion to teach is met (cf.…”