Consensus-based distributed linear support vector machines

Forero, Pedro A.; Cano, Alfonso; Giannakis, Georgios B.

doi:10.1145/1791212.1791218

Cited by 269 publications

(391 citation statements)

References 10 publications

Supporting

Mentioning

391

Contrasting

Order By: Relevance

“…Nevertheless, the method has been used for distributed SVM training in, e.g. [23]. The known convergence rates for ADMM are weaker than the more problem-tailored methods mentioned we study here, and the choice of the penalty parameter is often unclear in practice.…”

Section: Distributed Batch Solversmentioning

confidence: 99%

Distributed optimization with arbitrary local solvers

Konečný

Jäggi

et al. 2017

Optimization Methods and Software

172

166

View full text Add to dashboard Cite

With the growth of data and necessity for distributed optimization methods, solvers that work well on a single machine must be re-designed to leverage distributed computation. Recent work in this area has been limited by focusing heavily on developing highly specific methods for the distributed environment. These special-purpose methods are often unable to fully leverage the competitive performance of their well-tuned and customized single machine counterparts. Further, they are unable to easily integrate improvements that continue to be made to single machine methods. To this end, we present a framework for distributed optimization that both allows the flexibility of arbitrary solvers to be used on each (single) machine locally and yet maintains competitive performance against other state-of-the-art special-purpose distributed methods. We give strong primal-dual convergence rate guarantees for our framework that hold for arbitrary local solvers. We demonstrate the impact of local solver selection both theoretically and in an extensive experimental comparison. Finally, we provide thorough implementation details for our framework, highlighting areas for practical performance gains.Keywords: primal-dual algorithm; distributed computing; machine learning; convergence analysis 2010 Mathematics Subject Classification: 68W15; 68W20; 68W10; 68W40 MotivationRegression and classification techniques, represented in the general class of regularized loss minimization problems [71], are among the most central tools in modern big data analysis, machine learning, and signal processing. For these tasks, much effort from both industry and academia has gone into the development of highly tuned and customized solvers. However, with the massive growth of available datasets, major roadblocks still persist in the distributed setting, where data no longer fit in the memory of a single computer, and computation must be split across multiple machines in a network [3,7,12,18,22,29,32,34,37,46,52,62,64,67,78].On typical real-world systems, communicating data between machines is several orders of magnitude slower than reading data from main memory, e.g. when leveraging commodity hardware. Therefore when trying to translate existing highly tuned single machine solvers to the *Corresponding author. Email: takac.mt@gmail.com 814C. Ma et al. distributed setting, great care must be taken to avoid this significant communication bottleneck [26,74].While several distributed solvers for the problems of interest have been recently developed, they are often unable to fully leverage the competitive performance of their tuned and customized single machine counterparts, which have already received much more research attention. More importantly, it is unfortunate that distributed solvers cannot automatically benefit from improvements made to the single machine solvers, and therefore are forced to lag behind the most recent developments.In this paper, we make a step towards resolving these issues by proposing a general communication-efficient distribu...

show abstract

Section: Distributed Batch Solversmentioning

confidence: 99%

Distributed optimization with arbitrary local solvers

Konečný

Jäggi

et al. 2017

Optimization Methods and Software

172

166

View full text Add to dashboard Cite

show abstract

“…Current OSNS demands user to be system and policy administers for regulating their data, where users can restrict data sharing to a specific set of trusted users. In literature [6,7,8,9], several proposals of an access control scheme for OSNs have been introduced. But most of them were based on the property of trust.…”

Section: Related Workmentioning

confidence: 99%

Privacy Decision for Controlling Photo Sharing on Online Social Network

2017

IJAERD

View full text Add to dashboard Cite

show abstract

“…For a comprehensive review of consensus and gossip, the reader is directed to Garin and Schenato (2011) and the references therein. Examples of algorithms distributed using consensus are the Kalman filter (Olfati-Saber, 2005), detection (Bajovic, Jakovetic, Xavier, Sinopoli, & Moura, 2011), clustering (Forero, Cano, & Giannakis, 2011), support vector machines (Forero, Cano, & Giannakis, 2010), linear discriminant analysis (Valcarcel Macua, Belanovic, & Zazo, 2011), and many others. In this work we explore the application of consensus algorithms to SLGMs.…”

Section: Related Workmentioning

confidence: 99%

Distributed static linear Gaussian models using consensus

2012

View full text Add to dashboard Cite

Algorithms for distributed agreement are a powerful means for formulating distributed versions of existing centralized algorithms. We present a toolkit for this task and show how it can be used systematically to design fully distributed algorithms for static linear Gaussian models, including principal component analysis, factor analysis, and probabilistic principal component analysis. These algorithms do not rely on a fusion center, require only low-volume local (1-hop neighborhood) communications, and are thus efficient, scalable, and robust. We show how they are also guaranteed to asymptotically converge to the same solution as the corresponding existing centralized algorithms. Finally, we illustrate the functioning of our algorithms on two examples, and examine the inherent cost-performance tradeoff.

show abstract

Consensus-based distributed linear support vector machines

Cited by 269 publications

References 10 publications

Distributed optimization with arbitrary local solvers

Distributed optimization with arbitrary local solvers

Privacy Decision for Controlling Photo Sharing on Online Social Network

Distributed static linear Gaussian models using consensus

Contact Info

Product

Resources

About