Quadratic mixed integer programming and support vectors for deleting outliers in robust regression

Zioutas, G.; Pitsoulis, Leonidas; Avramidis, Antonios

doi:10.1007/s10479-008-0412-4

Cited by 16 publications

(14 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From another point of view, datasets with outliers pose a serious challenge in regression analysis and many solution techniques have been proposed (e.g. Panagopoulos et al 2019, andZioutas et al 2009). Mielke and Berry (1997) have proposed L 1 regression when errors are generated from fat-tailed or outlier-producing distributions, which are common in operations research.…”

Section: Introductionmentioning

confidence: 99%

Multi-criteria optimization in regression

Tsionas

2021

Ann Oper Res

View full text Add to dashboard Cite

In this paper, we consider standard as well as instrumental variables regression. Specification problems related to autocorrelation, heteroskedasticity, neglected non-linearity, unsatisfactory out-of-small performance and endogeneity can be addressed in the context of multi-criteria optimization. The new technique performs well, it minimizes all these problems simultaneously, and eliminates them for the most part. Markov Chain Monte Carlo techniques are used to perform the computations. An empirical application to NASDAQ returns is provided.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-criteria optimization in regression

Tsionas

2021

Ann Oper Res

View full text Add to dashboard Cite

show abstract

“…We impose two separate integer constraints on β and φ in (1), combining in a single framework the use of L 0 constraints for feature selection (Bertsimas et al 2016;Kenney et al 2018;Bertsimas and Van Parys 2020) and outlier detection (Bertsimas and Mazumder 2014;Zioutas et al 2009). In particular, we consider the following MIP formulation:…”

Section: Mip Formulationmentioning

confidence: 99%

“…In contrast to existing methodologies which extensively rely on heuristics, we propose a discrete and provably optimal approach to perform SFSOD, highlighting its connections with other approaches. The L 0 constraint has been used separately in the context of feature selection (Bertsimas et al 2016;Bertsimas and Van Parys 2020) and robust estimation (Zioutas et al 2009;Bertsimas and Mazumder 2014) -both of which can be formulated as a Mixed-Integer Program (MIP) and solved with optimality guarantees. We combine these two approaches into a novel formulation and take advantage of existing heuristics to provide effective big-M bounds and warm-starts to reduce the computational burden of MIP.…”

Section: Introductionmentioning

confidence: 99%

Simultaneous Feature Selection and Outlier Detection with Optimality Guarantees

Insolia¹,

Kenney²,

Chiaromonte³

et al. 2020

Preprint

View full text Add to dashboard Cite

Sparse estimation methods capable of tolerating outliers have been broadly investigated in the last decade. We contribute to this research considering high-dimensional regression problems contaminated by multiple mean-shift outliers which affect both the response and the design matrix. We develop a general framework for this class of problems and propose the use of mixed-integer programming to simultaneously perform feature selection and outlier detection with provably optimal guarantees. We characterize the theoretical properties of our approach, i.e. a necessary and sufficient condition for the robustly strong oracle property, which allows the number of features to exponentially increase with the sample size; the optimal estimation of the parameters; and the breakdown point of the resulting estimates. Moreover, we provide computationally efficient procedures to tune integer constraints and to warm-start the algorithm. We show the superior performance of our proposal compared to existing heuristic methods through numerical simulations and an application investigating the relationships between the human microbiome and childhood obesity.

show abstract

“…Interestingly enough, the nonsparse case of the robust subsets problem (1.2) (i.e., the standard low-dimensional LTS estimator) was formulated and solved as a mixed-integer program in Zioutas et al (2009), and their formulation can be considered (with certain small modifications) as a special case of our own mixed-integer program. The robust subsets estimator also bears close similarities to the high-dimensional robust regression estimator that appeared in Bhatia et al (2015).…”

Section: Robust Regressionmentioning

confidence: 99%

Robust subset selection

Thompson

2020

Preprint

View full text Add to dashboard Cite

The best subset selection (or "best subsets") estimator is a classic tool for sparse regression, and developments in mathematical optimization over the past decade have made it more computationally tractable than ever. Notwithstanding its desirable statistical properties, the best subsets estimator is susceptible to outliers and can break down in the presence of a single contaminated data point. To address this issue, we propose a robust adaption of best subsets that is highly resistant to contamination in both the response and the predictors. Our estimator generalizes the notion of subset selection to both predictors and observations, thereby achieving robustness in addition to sparsity. This procedure, which we call "robust subset selection" (or "robust subsets"), is defined by a combinatorial optimization problem for which we apply modern discrete optimization methods. We formally establish the robustness of our estimator in terms of the finite-sample breakdown point of its objective value. In support of this result, we report experiments on both synthetic and real data that demonstrate the superiority of robust subsets over best subsets in the presence of contamination. Importantly, robust subsets fares competitively across several metrics compared with popular robust adaptions of the Lasso.

show abstract

Quadratic mixed integer programming and support vectors for deleting outliers in robust regression

Cited by 16 publications

References 16 publications

Multi-criteria optimization in regression

Multi-criteria optimization in regression

Simultaneous Feature Selection and Outlier Detection with Optimality Guarantees

Robust subset selection

Contact Info

Product

Resources

About