Distance‐weighted discrimination

Marron, J. S.

doi:10.1002/wics.1345

Cited by 89 publications

(207 citation statements)

References 18 publications

Supporting

Mentioning

204

Contrasting

Order By: Relevance

“…For the classifiers based on Markov models considered in Section 2, the computation of the maximum likelihood estimates of parameters is fairly straight-forward; classifiers based on RD are computationally far less attractive than those based on Markov models. Methods based on support vector machines (SVM) (see Vapnik (1998);Hastie, Tibshirani, and Friedman (2009)) and distance weighted discrimination (DWD) (see Marron, Todd, and Ahn (2007);Qiao et al (2010)) are two well-known classification techniques available in the literature that are well equipped to deal with high-dimensional data sets, and it is appropriate to consider linear classifiers based on T x (k) using SVM and DWD as alternatives to RD.…”

Section: Other Linear Classifiersmentioning

confidence: 99%

Linear discriminant analysis of character sequences using occurrences of words

Dutta¹,

Chaudhuri²,

Ghosh³

2014

STAT SINICA

View full text Add to dashboard Cite

Classification of character sequences, where the characters come from a finite set, arises in disciplines such as molecular biology and computer science. For discriminant analysis of such character sequences, the Bayes classifier based on Markov models turns out to have class boundaries defined by linear functions of occurrences of words in the sequences. It is shown that for such classifiers based on Markov models with unknown orders, if the orders are estimated from the data using cross-validation, the resulting classifier has Bayes risk consistency under suitable conditions. Even when Markov models are not valid for the data, we develop methods for constructing classifiers based on linear functions of occurrences of words, where the word length is chosen by cross-validation. Such linear classifiers are constructed using ideas of support vector machines, regression depth, and distance weighted discrimination. We show that classifiers with linear class boundaries have certain optimal properties in terms of their asymptotic misclassification probabilities. The performance of these classifiers is demonstrated in various simulated and benchmark data sets. Key words and phrases:Bayes classifier, Markov and hidden Markov models, misclassification probability, order of a Markov model, V -fold cross-validation, word frequency.

show abstract

Section: Other Linear Classifiersmentioning

confidence: 99%

Linear discriminant analysis of character sequences using occurrences of words

Dutta¹,

Chaudhuri²,

Ghosh³

2014

STAT SINICA

View full text Add to dashboard Cite

show abstract

“…In a recent extensive numerical study by Fernández-Delgado et al (2014), the kernel SVM was shown to be one of the best among 179 commonly used classifiers. Marron et al (2007) invented a new classification algorithm named distance-weighted discrimination (DWD), which retains the elegant geometric interpretation of the SVM, resolves a 'data piling' issue and reveals competitive performance. Since then much work has been devoted to the development of DWD.…”

Section: Introductionmentioning

confidence: 99%

“…Since then much work has been devoted to the development of DWD. Readers are referred to Marron (2015) for an up-to-date list of work on DWD. However, DWD is still only known to a small group of researchers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Another Look at Distance-Weighted Discrimination

Wang

Zou

2017

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

Summary. Distance-weighted discrimination (DWD) is a modern margin-based classifier with an interesting geometric motivation. It was proposed as a competitor to the support vector machine (SVM). Despite many recent references on DWD, DWD is far less popular than the SVM, mainly because of computational and theoretical reasons. We greatly advance the current DWD methodology and its learning theory. We propose a novel thrifty algorithm for solving standard DWD and generalized DWD, and our algorithm can be several hundred times faster than the existing state of the art algorithm based on second-order cone programming. In addition, we exploit the new algorithm to design an efficient scheme to tune generalized DWD. Furthermore, we formulate a natural kernel DWD approach in a reproducing kernel Hilbert space and then establish the Bayes risk consistency of the kernel DWD by using a universal kernel such as the Gaussian kernel. This result solves an open theoretical problem in the DWD literature. A comparison study on 16 benchmark data sets shows that data-driven generalized DWD consistently delivers higher classification accuracy with less computation time than the SVM.

show abstract

“…In the literature, there are a large number of classifiers available, for example, linear discriminant analysis [1], support vector machines (SVMs) [2], Random Forests [3], distance-weighted discrimination (DWD) [4], and large margin unified machines (LUMs) [5]. The study by Hastie et al [6] provides a comprehensive review of many machine learning techniques.…”

Section: Introductionmentioning

confidence: 99%

Composite large margin classifiers with latent subclasses for heterogeneous biomedical data

Chen

Liu

Shen

et al. 2016

Statistical Analysis

View full text Add to dashboard Cite

High dimensional classification problems are prevalent in a wide range of modern scientific applications. Despite a large number of candidate classification techniques available to use, practitioners often face a dilemma of choosing between linear and general nonlinear classifiers. Specifically, simple linear classifiers have good interpretability, but may have limitations in handling data with complex structures. In contrast, general nonlinear classifiers are more flexible, but may lose interpretability and have higher tendency for overfitting. In this paper, we consider data with potential latent subgroups in the classes of interest. We propose a new method, namely the Composite Large Margin Classifier (CLM), to address the issue of classification with latent subclasses. The CLM aims to find three linear functions simultaneously: one linear function to split the data into two parts, with each part being classified by a different linear classifier. Our method has comparable prediction accuracy to a general nonlinear classifier, and it maintains the interpretability of traditional linear classifiers. We demonstrate the competitive performance of the CLM through comparisons with several existing linear and nonlinear classifiers by Monte Carlo experiments. Analysis of the Alzheimer’s disease classification problem using CLM not only provides a lower classification error in discriminating cases and controls, but also identifies subclasses in controls that are more likely to develop the disease in the future.

show abstract

Distance‐weighted discrimination

Cited by 89 publications

References 18 publications

Linear discriminant analysis of character sequences using occurrences of words

Linear discriminant analysis of character sequences using occurrences of words

Another Look at Distance-Weighted Discrimination

Composite large margin classifiers with latent subclasses for heterogeneous biomedical data

Contact Info

Product

Resources

About