Experiments with an extended tangent distance

Abstract

Abstract

Training Very Deep Networks via Residual Learning with Stochastic Input Shortcut Connections

“…It will be seen that with 54-hidden layers, our model achieves a state-of-the-art performance; that is, an error rate of 2.69%, surpassing the conventional ResNet (baseline model). In addition, Fig.3right shows the performance of the best proposed model (54 hidden layer S-ResNet) Invariant vector supports [20] 3.00 Neural network (LetNet) [21] 4.20 Sparse Large Margin Classifiers (SLMC) [22] 4.90 Incrementally Built Dictionary Learning (IBDL-C) [23] 3.99 Neural network + boosting [21] * 2.60 Tangent distance [24] * 2.50 Human performance [24] 2.50 Kernel density + virtual data [25] * 2.40 Kernel density + virtual data + classifier combination [25] * 2.20 Nearest neighbour [25] 5. Table 1.…”

Adjustable invariant features by partial Haar-integration

“…To address real-world applicability, we tested our approach on classifying the raw US-Postal-Service (USPS) Method Error rate [%] SVM, no invariance [9] 4.0 SVM, VSV-method [9] 3.2 TD + kernel densities [6] 2.4 Human Performance [12] 2.5 by padding out-of-image pixels with zero and performing bilinear interpolation.…”

Invariant kernel functions for pattern analysis and machine learning

“…The invariant distance computation is then based on minimizing the distance between the sets of transformed samples. Similar formalizations of such distances are widely available (Vasconcelos and Lippman 1998;Simard et al 1998;Keysers et al 2000). In particular this notion of invariant distance covers many specific examples in literature.…”

“…If in other cases the set T is composed of exponentially many combinations of transformations, which operate locally on independent parts of the pattern x, efficient computation can be performed by sequentially addressing the different object parts and their local transformations. An example for this is the IDM (Keysers et al 2000) for images, which can be evaluated in complexity growing linear in the number of pixels. If in other cases the assumption of linear representation of the sets T x holds, projection methods can be used to perform the exact minimization over infinitely many transformations very efficiently.…”

“…In particular this notion of invariant distance covers many specific examples in literature. In particular tangent distance (TD) (Simard et al 1993), the image distortion model (IDM) (Keysers et al 2000), general deformation models (Keysers et al 2004), dynamic time warping (DTW) (Rabiner and Juang 1993), Fréchet distance (Alt and Guibas 1999), invariant distances between point sets (Werman and Weinshall 1995) or two-sided manifold distance (Fitzgibbon and Zisserman 2003).…”