Online Metric Learning Methods Using Soft Margins and Least Squares Formulations

Abstract

Abstract. Online metric learning using margin maximization has been introduced as a way to learn appropriate dissimilarity measures in an efficient way when information as pairs of examples is given to the learning system in a progressive way. These schemes have several practical advantages with regard to global ones in which a training set needs to be processed. On the other hand, they may suffer from a poor performance depending on the quality of the examples and the particular tuning or other implementation details. This paper formulates several online metric learning alternatives using a passive-aggressive schema. A new formulation of the online problem using least squares is also introduced. The relative behavior of the different alternatives is studied and comparative experimentation is carried out to put forward the benefits and weaknesses of each alternative.

Passive–aggressive online distance metric learning and extensions

“…A slightly different alternative formulation using least squares [19] is also possible for the previous metric learning problem [12]. Instead of forcing a soft margin by penalizing the deviation from the ideal conditions, it is possible to force similar and dissimilar distance values to fall close to the "representative" values b − 1 and b + 1, respectively.…”

“…The experimentation setup in this work has been fixed as suggested in [5] and other previous preliminary studies [12]. Also, the Information theoretic metric learning algorithm (ITML) [5] has been considered in this work as a baseline for comparison purposes.…”

“…Also, the performance of the final solution may deviate significantly from the ideal (global) goal depending on the particular instances used in the last few iterations. Finally, parameter tuning gets considerably harder than in the batch case, according to recent preliminary results carried out in this context [12].…”

Comparative Evaluation of Batch and Online Distance Metric Learning Approaches Based on Margin Maximization

“…they are given sequentially or are obtained from a stream. Incremental and online methods [5], [10], [12] usually aim at optimizing a convenient criterion over a single instance (usually pairs of objects) which is made available for learning at every time step in the corresponding algorithm.…”