Piecewise-Linear Approximation for Feature Subset Selection in a Sequential Logit Model

Abstract

This paper concerns a method of selecting a subset of features for a sequential logit model. Tanaka and Nakagawa (2014) proposed a mixed integer quadratic optimization formulation for solving the problem based on a quadratic approximation of the logistic loss function. However, since there is a significant gap between the logistic loss function and its quadratic approximation, their formulation may fail to find a good subset of features. To overcome this drawback, we apply a piecewise-linear approximation to the logistic loss function. Accordingly, we frame the feature subset selection problem of minimizing an information criterion as a mixed integer linear optimization problem. The computational results demonstrate that our piecewise-linear approximation approach found a better subset of features than the quadratic approximation approach.

Feature Subset Selection for Ordered Logit Model via Tangent-Plane-Based Approximation

“…In line with Sato et al [22], we propose a computationally tractable MILO formulation for feature subset selection in the ordered logit model. We make use of tangent planes to approximate a bivariate nonlinear function involved in the ordered logit model.…”

“…To resolve this issue, Tanaka and Nakagawa [26] devised a mixed-integer quadratic optimization (MIQO) formulation based on a quadratic approximation of the nonlinear function. Sato et al [22] derived a mixed-integer linear optimization (MILO) formulation by applying a tangent-line-based approximation to the nonlinear function. They also showed that their MILO formulation offers better solution quality than the MIQO formulation.…”

Investigating consumers’ store-choice behavior via hierarchical variable selection

“…Although the stepwise method (Efroymson 1960), regularized/penalized regression (Tibshirani 1996), and metaheuristics (Yusta 2009) are practical approaches to variable selection, they do not necessarily find the best set of explanatory variables under the given goodness-of-fit measures. Hence, alternative approaches based on mixed-integer optimization (MIO) are now receiving considerable attention (Bertsimas and King in press;Konno and Yamamoto 2009;Maldonado et al 2014;Miyashiro and Takano 2015a,b;Sato et al 2016aSato et al , 2017Ustun and Rudin 2016;Wilson and Sahinidis in press) because these have the potential to provide the best set of explanatory variables with respect to several goodness-of-fit measures.…”

Best Subset Selection for Eliminating Multicollinearity

“…The mixed integer optimization (MIO) approach to subset selection has recently received much attention due to advances in algorithms and hardware [5,6,[20][21][22][27][28][29][30]. In contrast to heuristic algorithms, the MIO approach has the potential to provide the best subset of variables under a given goodness-of-fit measure.…”