Infinite-Label Learning with Semantic Output Codes

Abstract

We formalize a new statistical machine learning paradigm, called infinite-label learning, to annotate a data point with more than one relevant labels from a candidate set which pools both the finite labels observed at training and a potentially infinite number of previously unseen labels. The infinite-label learning fundamentally expands the scope of conventional multi-label learning and better meets the practical requirements in various real-world applications, such as image tagging, ads-query association, and article categorization. However, how can we learn a labeling function that is capable of assigning to a data point the labels omitted from the training set? To answer the question, we seek some clues from the recent works on zero-shot learning, where the key is to represent a class/label by a vector of semantic codes, as opposed to treating them as atomic labels. We validate the infinite-label learning by a PAC bound in theory and some empirical studies on both synthetic and real data.

Multi-label zero-shot human action recognition via joint latent ranking embedding

“…8.5 ± 0.6 30.7 ± 0.6 6.1 ± 0.6 50.0 ± 0.0 10.9 ± 0.9 5.4 ± 0.0 13.9 ± 0.0 5.1 ± 0.0 12.5 ± 0.0 7.2 ± 0.0 DSP [16] 15.9 ± 1. 5 of all five evaluation metrics under all three evaluation scenarios. Such results demonstrate that simply combining semantic representations of co-occurred multiple labels into one collective representation leads to catastrophic loss of semantic information, which is mainly responsible for the poor performance of DSP and ConSE in multi-label recognition.…”

“…Zero-shot learning (ZSL) has attracted much attention in recent years and provides a promising technique for recognizing a large number of classes without the need of the training data concerning all the classes. Very recently, [5] have formally shown that it is feasible to predict a collection of infinite unseen labels with a classifier learned on training data concerning only a number of labels in this collection or a subset of this collection, where multi-label ZSL is a special case in this socalled "infinite-label learning" paradigm. According to a ZSL taxonomy [15], existing ZSL approaches are divided into three categories, namely, direct mapping [16,17,18,19,20], model parameter transfer [21,22] and joint latent space learning [23,24,25,26,27,28,15,29].…”

“…For example, the famous Word2Vec semantic embedding is obtained by training a skip-gram neural network on the large-scale corpora, e.g., Google News dataset [39]. Such semantic representations are widely used in ZSL, e.g., [5,6,30,33,15]. Unlike label embedding obtained with external resources , co-occurrence of labels is yet another way to capture the relatedness between different labels, e.g., [32].…”

“…ZSL aims to recognize the instances belonging to novel classes which are not seen during training. It has been formally shown that under certain conditions, a ZSL system trained on a dataset of finite classes could be used to predict infinite number of classes unseen during the learning [5]. Under the ZSL framework, we merely need to collect and annotate training examples for a moderate set of training classes and expect that a large number of novel classes can be recognized via exploiting the semantic relationship between different human actions.…”

“…0.0 25.2 ± 0.0 16.6 ± 0.0 19.1 ± 0.0 17.7 ± 0.0 8.0 ± 0.0 12.6 ± 0.0 13.0 ± 0.0 7.4 ± 0.0 9.5 ± 0.0 ConSE[62] 16.7 ± 0.0 30.8 ± 0.0 21.1 ± 0.0 24.4 ± 0.0 22.7 ± 0.0 7.4 ± 0.0 14.6 ± 0.0 15.8 ± 0.0 9.0 ± 0.0 11.5 ± 0.0 COSTA[22] 21.5 ± 0.0 39.3 ± 0.0 29.5 ± 0.0 34.0 ± 0.0 31.6 ± 0.0 9.1 ± 0.0 18.6 ± 0.0 19.5 ± 0.0 11.2 ± 0.0 14.2 ± 0.0 Fast0Tag[6] 21.6 ± 1 5. 40.7 ± 0.6 28.7 ± 0.6 33.1 ± 0.7 30.7 ± 0.6 9.5 ± 0.0 23.4 ± 0.0 23.7 ± 0.1 13.6 ± 0.1 17.3 ± 0.1 Fast0Tag+ 23.2 ± 0.3 42.1 ± 0.5 30.4 ± 0.3 35.1 ± 0.3 32.6 ± 0.3 10.0 ± 0.0 24.3 ± 0.2 24.6 ± 0.3 14.1 ± 0.2 17.9 ± 0.2 Ours (RankNet) 32.7 ± 0.4 54.0 ± 0.5 39.1 ± 0.5 45.1 ± 0.5 41.9 ± 0.5 10.2 ± 0.1 24.9 ± 0.4 25.5 ± 0.4 14.6 ± 0.3 18.5 ± 0.3 Ours (Hinge) 33.8 ± 0.2 55.0 ± 0.3 39.7 ± 0.2 45.8 ± 0.2 42.5 ± 0.2 10.5 ± 0.1 25.2 ± 0.1 25.5 ± 0.3 14.6 ± 0.2 18.6 ± 0.2 Ours(Fusion) 34.1 ± 0.2 55.2 ± 0.2 39.7 ± 0.3 45.8 ± 0.4 42.5 ± 0.4 10.8 ± 0.1 25.8 ± 0.1 26.3 ± 0.2 15.0 ± 0.1 19.1 ± 0.1…”