Detecting relevant variables and interactions in supervised classification

Abstract

The widely used Support Vector Machine (SVM) method has shown to yield good results in Supervised Classification problems. When the interpretability is an important issue, then classification methods such as Classification Trees (CART) might be more attractive, since they are designed to detect the important predictor variables and, for each predictor variable, the critical values which are most relevant for classification. However, when interactions between variables strongly affect the class membership, CART may yield misleading information. Extending previous work of the authors, in this paper an SVM-based method is introduced. The numerical experiments reported show that our method is competitive against SVM and CART in terms of misclassification rates, and, at the same time, is able to detect critical values and variables interactions which are relevant for classification.

Feature selection for Support Vector Machines via Mixed Integer Linear Programming

“…Additionally, a low-dimensional representation allows a better interpretation of the classier. This is particularly important in some application elds like business analytics, since machine learning approaches are considered as black boxes by practitioners, and therefore they tend to be reticent to use these techniques (Carrizosa et al, 2011). The understanding of the process that generates the data is also of crucial importance in life sciences, e.g., the relevant genes that lead to a better discrimination in cancer prediction.…”

Interpretable support vector machines for functional data

“…In particular, L 1 -norm SVM (Bradley & Mangasarian, 1998;Carrizosa et al, 2010Carrizosa et al, , 2011Pedroso & Murata, 2001) and Fused SVM (Tibshirani et al, 2005;Rapaport et al, 2008) turn out to be particular cases of ISVMFD for particular choices of the derivatives.…”

“…Recently, a two-stage iterated method is proposed for credit decision making (Li et al, 2011), which combines feature selection and multi-criteria programming. In Carrizosa et al (2010Carrizosa et al ( , 2011, one-step SVM-based procedures are proposed to get the relevant variables and the relevant interactions between variables. Although one would expect classification rates to be deteriorated when looking for interpretable classifiers, the experiments in Carrizosa et al (2010Carrizosa et al ( , 2011 show that their proposals are competitive with SVM.…”

Supervised classification and mathematical optimization

“…Fourth, by focusing on a given predictor variable, the weights of the corresponding critical values can be used to plot a stepwise function that models how the variable influences the classifier. This approach has been extended to detect not only relevant variables but also relevant interactions between variables to classification [49].…”