Efficient Algorithms and Lower Bounds for Robust Linear Regression

Diakonikolas, Ilias; Kong, Weihao; Stewart, Alistair

doi:10.1137/1.9781611975482.170

Cited by 76 publications

(83 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In simple LRs two variables are used for finding the predictive function i.e. the predictor variable which the independent variable and the criterion variable which is the dependant variable [35]. We have exploited LR algorithm for forecasting the routing parameters as the predictor variables to generate a predictive model based on observed routing dataset.…”

Section: A Linear Regressionmentioning

confidence: 99%

See 1 more Smart Citation

Prediction of Scenarios for Routing in MANETs Based on Expanding Ring Search and Random Early Detection Parameters Using Machine Learning Techniques

2021

View full text Add to dashboard Cite

Routing protocols in Mobile Ad Hoc Networks (MANETs) play a pivotal role in ensuring quality of service (QoS) and improving network performance. Selection of optimal routing protocol and suitable parameters for a given network scenario is a major task that ultimately affects the behavior of network. This work exploits machine learning (ML) techniques for the selection of adequate routing parameters and protocol by regression of parameters in given network scenario to ensure optimal performance. The network is trained based on parametric setup of expanding ring search mechanism (ERS) and random early detection (RED) technique to estimate network throughput, end to end (E2E) delay and packets delivery ratio (PDR) and is tested via wide-ranging simulations in varying network topologies. Both RED and ERS mechanisms are aimed to control link and node level congestion in the reactive routing protocols and our aim is to select the best suited parameters for given network topologies based on ERS and RED parametric setups and improve performance for ensuring QoS. ML algorithms are trained and tested for their performance in varying network topologies. We have exploited these models with best performance for ERS and RED based routing in given topological arrangements. The performance of the ML algorithms is evaluated on the basis of root mean squared error (RMSE) and mean absolute error (MAE) for regression settings. Prediction models with up to par RMSE and MAE out-turns are attained and exploited for selection of suitable ERS and RED parameters and routing protocols in order to ensure the QoS for given network scenario. Variants of standard routing protocols are devised based on their performance and the ML techniques are exploited for prediction of QoS parameters to decide on the optimal variant that attains significant improvement in performance. Results are shown to confirm that considerable improvement in QoS is attained. INDEX TERMS Ad hoc multi hop wireless networks, congestion control, expanding ring search, machine learning, mobile ad hoc networks, on demand routing protocols, random early detection, regression, quality of service.

show abstract

Section: A Linear Regressionmentioning

confidence: 99%

“…Similarly irrelevant attributes to the output variable are also removed with feature selection methods. In order to reduce the complexity of the model ridge regularization technique is used that averts any coefficient to reach high value by reducing the absolute sum square of the learned coefficients [35].…”

Section: A Linear Regressionmentioning

confidence: 99%

Prediction of Scenarios for Routing in MANETs Based on Expanding Ring Search and Random Early Detection Parameters Using Machine Learning Techniques

2021

View full text Add to dashboard Cite

show abstract

“…Finally, we would like to add that moment matching was used in prior work to derive statistical query lower bounds for mixture distributions [17,20], to defend against adversarial examples [15], for robust statistics [21] and in other settings. Their proofs required different constructions from the one appearing in this paper.…”

Section: Related Workmentioning

confidence: 99%

Interaction is necessary for distributed learning with privacy or communication constraints

Dagan

Feldman

2020

Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing

View full text Add to dashboard Cite

Local differential privacy (LDP) is a model where users send privatized data to an untrusted central server whose goal it to solve some data analysis task. In the non-interactive version of this model the protocol consists of a single round in which a server sends requests to all users then receives their responses. This version is deployed in industry due to its practical advantages and has attracted significant research interest. Our main result is an exponential lower bound on the number of samples necessary to solve the standard task of learning a large-margin linear separator in the non-interactive LDP model. Via a standard reduction this lower bound implies an exponential lower bound for stochastic convex optimization and specifically, for learning linear models with a convex, Lipschitz and smooth loss. These results answer the questions posed by Smith, Thakurta, and Upadhyay (IEEE Symposium on Security and Privacy 2017) and Daniely and Feldman (NeurIPS 2019). Our lower bound relies on a new technique for constructing pairs of distributions with nearly matching moments but whose supports can be nearly separated by a large margin hyperplane. These lower bounds also hold in the model where communication from each user is limited and follow from a lower bound on learning using non-adaptive statistical queries.

show abstract

“…Ideally, we wish to find a robust algorithm with breakdown point asymptotically convergent to 1 while being consistent under adaptive adversarial corruption. However, it is impossible to satisfy all the three requirements Diakonikolas et al [2018]. A popular approach is to use 1 -norm in the loss function.…”

Section: Related Workmentioning

confidence: 99%

Robust Gaussian Process Regression for Real-Time High Precision GPS Signal Enhancement

Lin

Song

Qian

et al. 2019

Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery &Amp; Data Mining

View full text Add to dashboard Cite

Satellite-based positioning system such as GPS often suffers from large amount of noise that degrades the positioning accuracy dramatically especially in real-time applications. In this work, we consider a data-mining approach to enhance the GPS signal. We build a large-scale high precision GPS receiver grid system to collect real-time GPS signals for training. The Gaussian Process (GP) regression is chosen to model the vertical Total Electron Content (vTEC) distribution of the ionosphere of the Earth. Our experiments show that the noise in the real-time GPS signals often exceeds the breakdown point of the conventional robust regression methods resulting in sub-optimal system performance. We propose a three-step approach to address this challenge. In the first step we perform a set of signal validity tests to separate the signals into clean and dirty groups. In the second step, we train an initial model on the clean signals and then reweigting the dirty signals based on the residual error. A final model is retrained on both the clean signals and the reweighted dirty signals. In the theoretical analysis, we prove that the proposed three-step approach is able to tolerate much higher noise level than the vanilla robust regression methods if two reweighting rules are followed. We validate the superiority of the proposed method in our real-time high precision positioning system

show abstract

Efficient Algorithms and Lower Bounds for Robust Linear Regression

Cited by 76 publications

References 16 publications

Prediction of Scenarios for Routing in MANETs Based on Expanding Ring Search and Random Early Detection Parameters Using Machine Learning Techniques

Prediction of Scenarios for Routing in MANETs Based on Expanding Ring Search and Random Early Detection Parameters Using Machine Learning Techniques

Interaction is necessary for distributed learning with privacy or communication constraints

Robust Gaussian Process Regression for Real-Time High Precision GPS Signal Enhancement

Contact Info

Product

Resources

About