Best subset binary prediction

Abstract

We consider a variable selection problem for the prediction of binary outcomes. We study the best subset selection procedure by which the explanatory variables are chosen by maximizing Manski (1975Manski ( , 1985's maximum score type objective function subject to a constraint on the maximal number of selected variables. We show that this procedure can be equivalently reformulated as solving a mixed integer optimization (MIO) problem, which enables computation of the exact or an approximate solution with a definite approximation error bound. In terms of theoretical results, we obtain nonasymptotic upper and lower risk bounds when the dimension of potential covariates is possibly much larger than the sample size. Our upper and lower risk bounds are minimax rate-optimal when the maximal number of selected variables is fixed and does not increase with the sample size. We illustrate usefulness of the best subset binary prediction approach via Monte Carlo simulations and an empirical application of the work-trip transportation mode choice. * We are grateful to the co-editor, Jianqing Fan, an associate editor and three anonymous referees for constructive comments and suggestions. We also thank

Exact computation of GMM estimators for instrumental variable quantile regression models

“…Modern mixed integer optimization solvers employ branch‐and‐bound type algorithms which maintain along the solution process both the feasible solutions and lower bounds on the optimal objective function value. For computationally demanding applications, this feature enables us to solve for an approximate IVQR GMM estimator with a guaranteed approximation error bound, thus facilitating the design of an early stopping rule as described in Chen and Lee (, Section 4.3). Development of such a theoretically justified early stopping rule for the IVQR estimation problem is therefore a useful further research topic.…”

“…For classic texts on the MIO methodology and applications, see Nemhauser and Wolsey () and Bertsimas and Weismantel (). See also Florios and Skouras (), Bilias, Florios, and Skouras (), Kitagawa and Tetenov (), Bertsimas, King, and Mazumder () and Chen and Lee () for related but distinct work on solving nonconvex optimization problems in statistics and econometrics via the MIO approach.…”

Sparse HP filter: Finding kinks in the COVID-19 contact rate

“…We build the sparse HP filter by drawing on the recent literature that uses an -constraint or -penalty (see, e.g. Bertsimas et al, 2016 , Chen and Lee, 2018 , Chen and Lee, 2020 , Huang et al, 2018 ).…”

“…As an alternative to trend filtering, we may exploit Assumption 3 and consider an -constrained version of trend flitering: The formulation in (3.4) is related to the method called best subset selection (see, e.g. Bertsimas et al, 2016 , Chen and Lee, 2018 ). It requires only the input of .…”

Exact computation of GMM estimators for instrumental variable quantile regression models

“…For classic texts on the MIO methodology and applications, see Nemhauser and Wolsey (1999) and Bertsimas and Weismantel (2005). See also Florios and Skouras (2008), Bilias, Florios, and Skouras (2013), Kitagawa and Tetenov (2015), Bertsimas, King, and Mazumder (2016) and Chen and Lee (2016) for related but distinct work on solving non-convex optimization problems in statistics and econometrics via the MIO approach.…”