Attributes for Classifier Feedback

Abstract

Abstract. Traditional active learning allows a (machine) learner to query the (human) teacher for labels on examples it finds confusing. The teacher then provides a label for only that instance. This is quite restrictive. In this paper, we propose a learning paradigm in which the learner communicates its belief (i.e. predicted label) about the actively chosen example to the teacher. The teacher then confirms or rejects the predicted label. More importantly, if rejected, the teacher communicates an explanation for why the learner's belief was wrong. This explanation allows the learner to propagate the feedback provided by the teacher to many unlabeled images. This allows a classifier to better learn from its mistakes, leading to accelerated discriminative learning of visual concepts even with few labeled images. In order for such communication to be feasible, it is crucial to have a language that both the human supervisor and the machine learner understand. Attributes provide precisely this channel. They are human-interpretable mid-level visual concepts shareable across categories e.g. "furry", "spacious", etc. We advocate the use of attributes for a supervisor to provide feedback to a classifier and directly communicate his knowledge of the world. We employ a straightforward approach to incorporate this feedback in the classifier, and demonstrate its power on a variety of visual recognition scenarios such as image classification and annotation. This application of attributes for providing classifiers feedback is very powerful, and has not been explored in the community. It introduces a new mode of supervision, and opens up several avenues for future research.

Interactively Guiding Semi-Supervised Clustering via Attribute-Based Explanations

“…computer vision tasks like image search [17,18] and classification [19,20,15,2]. In this work, we show that attributes can be used for achieving more accurate clusterings when using semi-supervised clustering algorithms.…”

“…Attributes are mid-level concepts that have been extensively used for a variety of tasks in computer vision [15,23,2,24,22,25,18,17,19,20,26,16]. The vocabulary of attributes can be pre-defined or it can be discovered [27,28].…”

“…The vocabulary of attributes can be pre-defined or it can be discovered [27,28]. Recently, attributes have been used as a mode of communication between humans and machines [17,19,20]. Donahue and Grauman [20] use spatial annotations and binary attributes to allow an annotator to provide an explanation for a particular class label.…”

“…In our approach, we use attributes to propagate constraints to pairs of images in an interactive semi-supervised clustering setting. Parkash and Parikh [19] use relative attributes feedback (e.g., "this image is too open to be a forest") to identify additional negative examples from an unlabelled pool of images to train a classifier. Attributes have also been used for fine-grained classification with a human-in-the-loop who conveys domain knowledge by answering attribute-based questions at test time to aid the machine [25].…”

Active Learning for Human Pose Estimation

“…tools for efficient video annotation [53] and object labelling [34]. Methods that intelligently design the query space [39,32,30] also share the spirit of reducing annotation effort. Other works have looked into active learning schemes that query for multiple types of annotator feedback [50,4,43].…”