Centroids Triplet Network and Temporally-Consistent Embeddings for In-Situ Object Recognition

Abstract

This work proposes learning to recognize objects from a small number of training examples collected and deployed in-situ. That is, from data collected where the objects are commonly placed or being used, perhaps after first encountering them, the learning algorithm immediately is able to recognize them again. We refer to this methodology as in-situ learning, and it opposes to the conventional methodology of using complex data acquisition mechanisms, such as rotating tables or synthetic data, to build a large-scale dataset for training convolutional neural networks (ConvNets). To learn in-situ, we propose a novel loss function that generates discriminative features for known and unseen objects, by utilizing a regularization term that reduces the distance between features and their manifold centroid. Additionally, we propose a temporal filter that is particularly useful to quickly react to appearing objects on the scene, which depending on the distance between neighboring video-frame features, it applies a weighted average between the current and the previous frame. Our framework achieves state-of-the-art accuracy for in-situ and on-the-fly learning, for the case of known objects achieves an average increase in accuracy of 3.01%, an increase of 3.3% for novel objects, and an average increase of 7.07% for the combined case, compared with the closest baseline. Utilizing the temporal filtering, led to a further increase in accuracy against nuisances of 7.32% for the known and novels objects case.

On the Unreasonable Effectiveness of Centroids in Image Retrieval

“…Currently most works use comparative/ranking losses and the Triplet Loss is one of the most widely used approaches. However, state-of-the-art solutions often combine a comparative loss with auxiliary losses such as classification or center loss [5,7,12,13,16]. * Both authors contributed equally to this research.…”

“…A centroid approach results in one embedding per item, decreasing both memory and storage requirements. There are a number of approaches investigating the prototype/centroid formulation and their main advantages are as follows: 1) Lower computational cost [2,16], of even linear complexity instead of cubic [2]; 2) Higher robustness to outliers and noisy labels [16,18]; 3) Faster training [11]; 4) Comparable or better performance than the standard point-to-point triplet loss [5,11,16].…”