Comparison of classifier methods: a case study in handwritten digit recognition

Bottou, Léon; Cortes, Corinna; Denker, John S.; Drucker, Harris; Guyon, Isabelle; Jackel, L. D.; LeCun, Yann; Müller, Urs; Sackinger, E.; Simard, Patrice; Vapnik, Vladimir

doi:10.1109/icpr.1994.576879

Cited by 463 publications

(237 citation statements)

References 7 publications

Supporting

Mentioning

230

Contrasting

Unclassified

Order By: Relevance

“…For a comprehensive approach to the related least squares support vector machines see , where geometric and statistical interpretations as well as the link with the Fischer discriminant analysis are given. It is the purpose of this work to apply this simple 2-class PSVM classifier to k-category classification by using a one-from-rest (OFR) separation for each class (Bottou et al, 1994). However, due to the fact that the number of points belonging to one class is usually much smaller than the number of points in the union of the remaining classes, the resulting two-class problems are very unbalanced.…”

Section: Introductionmentioning

confidence: 99%

“…In contrast, other one-from-the-rest and SVM k-class classifiers (Bottou et al, 1994;Bennett & Mangasarian, 1993;Bredensteiner & Bennett, 1999) require the solution of either a large single or k smaller quadratic or linear programs that need specialized optimization codes such as CPLEX (1992). On the other hand, obtaining a linear or nonlinear PSVM classifier as we propose here, requires nothing more sophisticated than solving k systems of linear equations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multicategory Proximal Support Vector Machine Classifiers

2005

View full text Add to dashboard Cite

Abstract. Given a dataset, each element of which labeled by one of k labels, we construct by a very fast algorithm, a k-category proximal support vector machine (PSVM) classifier. Proximal support vector machines and related approaches (Fung & Mangasarian, 2001; can be interpreted as ridge regression applied to classification problems (Evgeniou, Pontil, & Poggio, 2000). Extensive computational results have shown the effectiveness of PSVM for two-class classification problems where the separating plane is constructed in time that can be as little as two orders of magnitude shorter than that of conventional support vector machines. When PSVM is applied to problems with more than two classes, the well known one-from-the-rest approach is a natural choice in order to take advantage of its fast performance. However, there is a drawback associated with this one-from-the-rest approach. The resulting two-class problems are often very unbalanced, leading in some cases to poor performance. We propose balancing the k classes and a novel Newton refinement modification to PSVM in order to deal with this problem. Computational results indicate that these two modifications preserve the speed of PSVM while often leading to significant test set improvement over a plain PSVM one-from-the-rest application. The modified approach is considerably faster than other one-from-the-rest methods that use conventional SVM formulations, while still giving comparable test set correctness.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Multicategory Proximal Support Vector Machine Classifiers

2005

View full text Add to dashboard Cite

show abstract

“…Schölkopf and Smola [28] described the widely used multi-classification SVM techniques: oneversus-rest, pair-wise classification, error-correcting output codes and the multi-classification objective functions. The one-versus-the-rest also referred to as one-against-all (OAA) is probably the earliest SVM multi-class implementation [3]. It constructs c binary SVM classifiers where c is the number of classes.…”

Section: Svm Principles and Related Studiesmentioning

confidence: 99%

Dynamic classification for video stream using support vector machine

Awad

Motai

2008

Applied Soft Computing

View full text Add to dashboard Cite

A dynamic classification using the support vector machine (SVM) technique is presented in this paper as a new 'incremental' framework for multiple-classifying video stream data. The contribution of this study is the derivation of a unique, fast and simple to implement technique that allows multi-classification of behavioral motions based on an adaptation of the Least Square SVM (LS-SVM) formulation. This dynamic approach leads to an extension of SVM beyond its current static image-based learning methodologies. The proposed incremental multi-classification method is applied to video stream data, which consists of an articulated humanoid model monitored by a surveillance camera. The initial supervised off-line learning phase is followed by a visual behavior data acquisition and proposed incremental learning phase. The resulting error rate and the confidence level for the proposed technique demonstrate its validity and merits in articulated motion learning. Furthermore, the enabled online learning allows an adaptive domain knowledge insertion and provides the advantage of reducing both the model training time and the information storage requirements of the overall system which are both essential in dynamic soft computing applications.

show abstract

“…There are also hundreds of them being handled in the English OCR, if touched English letters are considered as separate class types from untouched letters. Since SVM training deals with one pair of class types at a time, we need to train l(l-1)/2 one-against-one (1A1) classifiers (Kerr et al [1]) or l one-against-others (1AO) classifiers (Bottou et al [2]), where l is the number of class types. Such a gigantic collection of classifiers not only poses a problem to the training but also to the testing of SVMs.…”

Section: Introductionmentioning

confidence: 99%