VISER: Visual Self-Regularization

Abstract

In this work, we propose the use of large set of unlabeled images as a source of regularization data for learning robust visual representation. Given a visual model trained by a labeled dataset in a supervised fashion, we augment our training samples by incorporating large number of unlabeled data and train a semi-supervised model. We demonstrate that our proposed learning approach leverages an abundance of unlabeled images and boosts the visual recognition performance which alleviates the need to rely on large labeled datasets for learning robust representation. To increment the number of image instances needed to learn robust visual models in our approach, each labeled image propagates its label to its nearest unlabeled image instances. These retrieved unlabeled images serve as local perturbations of each labeled image to perform Visual Self-Regularization (VISER). To retrieve such visual self regularizers, we compute the cosine similarity in a semantic space defined by the penultimate layer in a fully convolutional neural network. We use the publicly available Yahoo Flickr Creative Commons 100M dataset as the source of our unlabeled image set and propose a distributed approximate nearest neighbor algorithm to make retrieval practical at that scale. Using the labeled instances and their regularizer samples we show that we significantly improve object categorization and localization performance on the MS COCO and Visual Genome datasets where objects appear in context. * Work was done while the author was an intern at Flickr, Yahoo Research.

DIViS: Domain Invariant Visual Servoing for Collision-Free Goal Reaching

“…Our Semantic Net is built and trained based on [24]. We pre-train our Semantic Net on the MS-COCO object categories [34] with a weakly supervised object localization setup similar to [24].…”

“…Our Semantic Net is built and trained based on [24]. We pre-train our Semantic Net on the MS-COCO object categories [34] with a weakly supervised object localization setup similar to [24]. We use the penultimate layer of the fully convolutional neural network of [24] to encode visual semantics in the spatial image space.…”

“…We pre-train our Semantic Net on the MS-COCO object categories [34] with a weakly supervised object localization setup similar to [24]. We use the penultimate layer of the fully convolutional neural network of [24] to encode visual semantics in the spatial image space. Semantic Net describes each RGB image input via an N × N × 2048 map which we denote it as S(I) and is shown by yellow box in Figure 1.…”

“…To do that, we mask the last layer of the Semantic Net with the category id l to produce S l (I) and use it in lieu of the correlation map φ θs (I, l) = S l (I) which will be fed into our policy module. Given the fact that our Semantic Net is trained with real images and is robust to noisy samples [24], our approach for computing the semantic correlation map as an input to the policy network provides domain invariant representations which we will empirically show to work well in real world scenarios. In addition to that, our Collision Net is also domain invariant as it is pre-trained via domain randomization technique [49].…”