Multi-Label Learning from Single Positive Labels

Abstract

Predicting all applicable labels for a given image is known as multi-label classification. Compared to the standard multi-class case (where each image has only one label), it is considerably more challenging to annotate training data for multi-label classification. When the number of potential labels is large, human annotators find it difficult to mention all applicable labels for each training image. Furthermore, in some settings detection is intrinsically difficult e.g. finding small object instances in high resolution images. As a result, multi-label training data is often plagued by false negatives. We consider the hardest version of this problem, where annotators provide only one relevant label for each image. As a result, training sets will have only one positive label per image and no confirmed negatives. We explore this special case of learning from missing labels across four different multi-label image classification datasets for both linear classifiers and end-to-end finetuned deep networks. We extend existing multi-label losses to this setting and propose novel variants that constrain the number of expected positive labels during training. Surprisingly, we show that in some cases it is possible to approach the performance of fully labeled classifiers despite training with significantly fewer confirmed labels.

Spatial Consistency Loss for Training Multi-Label Classifiers from Single-Label Annotations

“…However, unannotated positives can deteriorate performance. Ignoring the unannotated classes in the loss function can alleviate this issue [12], but this is inapplicable when the annotations only contain positives [8]. Asymmetric loss design can help handle missing labels beyond the BCE loss [54].…”

“…Prior works consider single-positive labels in the single-label setting [8,54], as a combination of single-label learning [37,11,20] and positive-unlabeled learning [10,1]. [8] propose to go beyond label smoothing [42,47] to deal with the label noise introduced by false negative labels their regularized online label estimation (ROLE) method estimates the missing labels in an online fashion, by jointly optimizing a label estimator and image classifier, the output of the former serving as ground truth of the latter. We also propose a method to mine annotations expected to be positives in section 3.5 by keeping running averages of the class specific scores on the training set images, and show that this simpler method performs similarly to ROLE in section 4.…”

“…We state the problem of multi-label classification with partially annotated labels similarly to [8]. Our goal is to learn a mapping from an image x n to the indicator vector y n ∈ {0, 1} L of the classes contained in the image, L being the number of classes.…”

“…One simple strategy to handle single-positive labels is to assume that all unknown labels are negatives. This leads to the assume negative (AN) loss function [8] L…”

“…We use MS-COCO 2014 [30] and Pascal VOC 2012 [13] as benchmarks for multi-label classification. In order to test our contributions, we use the code shared by [8] to simulate a single-positive annotated setting, and reproduce their train, validation and test samples. In this setup, 20% of the original train data is first set aside for validation.…”