Visual recognition in the wild by sampling deep similarity functions

Abstract

Recognising relevant objects or object states in its environment is a basic capability for an autonomous robot. The dominant approach to object recognition in images and range images is classification by supervised machine learning, nowadays mostly with deep convolutional neural networks (CNNs). This works well for target classes whose variability can be completely covered with training examples. However, a robot moving in the wild, i.e., in an environment that is not known at the time the recognition system is trained, will often face domain shift: the training data cannot be assumed to exhaustively cover all the within-class variability that will be encountered in the test data. In that situation, learning is in principle possible, since the training set does capture the defining properties, respectively dissimilarities, of the target classes. But directly training a CNN to predict class probabilities is prone to overfitting to irrelevant correlations between the class labels and the specific subset of the target class that is represented in the training set. We explore the idea to instead learn a Siamese CNN that acts as similarity function between pairs of training examples. Class predictions are then obtained by measuring the similarities between a new test instance and the training samples. We show that the CNN embedding correctly recovers the relative similarities to arbitrary class exemplars in the training set. And that therefore few, randomly picked training exemplars are sufficient to achieve good predictions, making the procedure efficient.

Visual inspection via a global-to-local optimization method for agarwood sticks

“…To automatically and online inspect the burned agarwood sticks, a machine vision strategy is proposed in this paper, which involves a designed image acquisition system and a global-to-local optimization scheme. First, the target dissimilarity function [14][15][16] is introduced to define a dissimilarity coefficient by means of the attributes of the connected domains extracted from the agarwood stick image, which is utilized to coarsely localize the carbon line. Then, the threshold is adaptively determined based on the grayscale characteristics of image patches of the coarsely localized carbon line region, which is utilized for extracting the contour of the carbon line region.…”

Visual inspection via a global-to-local optimization method for agarwood sticks

“…To automatically and online inspect the burned agarwood sticks, a machine vision strategy is proposed in this paper, which involves a designed image acquisition system and a global-to-local optimization scheme. First, the target dissimilarity function [14][15][16] is introduced to define a dissimilarity coefficient by means of the attributes of the connected domains extracted from the agarwood stick image, which is utilized to coarsely localize the carbon line. Then, the threshold is adaptively determined based on the grayscale characteristics of image patches of the coarsely localized carbon line region, which is utilized for extracting the contour of the carbon line region.…”