Sparse Regression: Scalable Algorithms and Empirical Performance

Bertsimas, Dimitris; Pauphilet, Jean; Parys, Bart Van

doi:10.1214/19-sts701

Cited by 40 publications

(60 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A future direction of study will be to develop an efficient algorithm specialized for solving our MIQO problem. We are now working on extending several MIO-based high-performance algorithms [ 24 , 48 , 49 ] to sparse Poisson regression. Another direction of future research is to improve the performance of our methods for selecting tangent lines.…”

Section: Discussionmentioning

confidence: 99%

Sparse Poisson regression via mixed-integer optimization

2021

View full text Add to dashboard Cite

We present a mixed-integer optimization (MIO) approach to sparse Poisson regression. The MIO approach to sparse linear regression was first proposed in the 1970s, but has recently received renewed attention due to advances in optimization algorithms and computer hardware. In contrast to many sparse estimation algorithms, the MIO approach has the advantage of finding the best subset of explanatory variables with respect to various criterion functions. In this paper, we focus on a sparse Poisson regression that maximizes the weighted sum of the log-likelihood function and the L2-regularization term. For this problem, we derive a mixed-integer quadratic optimization (MIQO) formulation by applying a piecewise-linear approximation to the log-likelihood function. Optimization software can solve this MIQO problem to optimality. Moreover, we propose two methods for selecting a limited number of tangent lines effective for piecewise-linear approximations. We assess the efficacy of our method through computational experiments using synthetic and real-world datasets. Our methods provide better log-likelihood values than do conventional greedy algorithms in selecting tangent lines. In addition, our MIQO formulation delivers better out-of-sample prediction performance than do forward stepwise selection and L1-regularized estimation, especially in low-noise situations.

show abstract

Section: Discussionmentioning

confidence: 99%

Sparse Poisson regression via mixed-integer optimization

2021

View full text Add to dashboard Cite

show abstract

“…Despite its computational hardness, many practically useful algorithms have been proposed to solve 0 -regularized ERM, while the statistical properties beyond least squares regression are much less studied. We refer to [4] and [19] for two comprehensive survey articles on 0 -regularized regression methods.…”

Section: Related Literaturementioning

confidence: 99%

Iteratively reweighted ℓ1-penalized robust regression

Pan

Sun

Zhou

2021

Electron. J. Statist.

View full text Add to dashboard Cite

This paper investigates tradeoffs among optimization errors, statistical rates of convergence and the effect of heavy-tailed errors for high-dimensional robust regression with nonconvex regularization. When the additive errors in linear models have only bounded second moments, we show that iteratively reweighted 1 -penalized adaptive Huber regression estimator satisfies exponential deviation bounds and oracle properties, including the oracle convergence rate and variable selection consistency, under a weak beta-min condition. Computationally, we need as many as O(log s + log log d) iterations to reach such an oracle estimator, where s and d denote the sparsity and ambient dimension, respectively. Extension to a general class of robust loss functions is also considered. Numerical studies lend strong support to our methodology and theory.

show abstract

“…Bertsimas and Van Parys [5] developed a cutting-plane algorithm for the exact sparse regression problem and solved the problems with 100-thousand sample sizes and features within 10 minutes. In [4], the authors compared the out-ofsample accuracy and false detection rate for five methods under six noise/correlation scenarios. Based on extensive experiments, they demonstrated that Lasso performs poorly in low noise settings and is comparable with other methods as noise increases which explains the robustness aspect of Lasso.…”

Section: For a Linear Regression Problem ( ) =mentioning

confidence: 99%

Optimal Decision Tree for Cycle Time Prediction and Allowance Determination

Hsu

2021

IEEE Access

View full text Add to dashboard Cite

The due-date quotation is a key performance indicator for managing customer orders which would influence customer acceptance and/or the future potential lateness penalty. The production cycle time and allowance time are added and used as the due date of order. The objective is to maximize the hit rate which is the percentage of the orders fulfilled within the time limit of quoted due date. Under the framework of supervised machine learning, we explore the new developments in feature selection and the optimal decision tree to predict cycle time by using mixed-integer optimization. Cycle time allowance could be added to the predicted cycle time or incorporated in an optimization problem as a managerial decision variable. Case studies are used to demonstrate the effectiveness of this approach, and their performances are comparable to the other popular ensemble tree approaches, such as random forests and gradient boosting. INDEX TERMS Allowance determination, cycle time prediction, gradient boosting, hit rate, local search in a decision tree, mixed-integer optimization, optimal decision tree, random forests.

show abstract

Sparse Regression: Scalable Algorithms and Empirical Performance

Cited by 40 publications

References 39 publications

Sparse Poisson regression via mixed-integer optimization

Sparse Poisson regression via mixed-integer optimization

Iteratively reweighted ℓ1-penalized robust regression

Optimal Decision Tree for Cycle Time Prediction and Allowance Determination

Contact Info

Product

Resources

About