With the growth of data and necessity for distributed optimization methods, solvers that work well on a single machine must be re-designed to leverage distributed computation. Recent work in this area has been limited by focusing heavily on developing highly specific methods for the distributed environment. These special-purpose methods are often unable to fully leverage the competitive performance of their well-tuned and customized single machine counterparts. Further, they are unable to easily integrate improvements that continue to be made to single machine methods. To this end, we present a framework for distributed optimization that both allows the flexibility of arbitrary solvers to be used on each (single) machine locally and yet maintains competitive performance against other state-of-the-art special-purpose distributed methods. We give strong primal-dual convergence rate guarantees for our framework that hold for arbitrary local solvers. We demonstrate the impact of local solver selection both theoretically and in an extensive experimental comparison. Finally, we provide thorough implementation details for our framework, highlighting areas for practical performance gains.Keywords: primal-dual algorithm; distributed computing; machine learning; convergence analysis 2010 Mathematics Subject Classification: 68W15; 68W20; 68W10; 68W40 MotivationRegression and classification techniques, represented in the general class of regularized loss minimization problems [71], are among the most central tools in modern big data analysis, machine learning, and signal processing. For these tasks, much effort from both industry and academia has gone into the development of highly tuned and customized solvers. However, with the massive growth of available datasets, major roadblocks still persist in the distributed setting, where data no longer fit in the memory of a single computer, and computation must be split across multiple machines in a network [3,7,12,18,22,29,32,34,37,46,52,62,64,67,78].On typical real-world systems, communicating data between machines is several orders of magnitude slower than reading data from main memory, e.g. when leveraging commodity hardware. Therefore when trying to translate existing highly tuned single machine solvers to the *Corresponding author. Email: takac.mt@gmail.com 814C. Ma et al. distributed setting, great care must be taken to avoid this significant communication bottleneck [26,74].While several distributed solvers for the problems of interest have been recently developed, they are often unable to fully leverage the competitive performance of their tuned and customized single machine counterparts, which have already received much more research attention. More importantly, it is unfortunate that distributed solvers cannot automatically benefit from improvements made to the single machine solvers, and therefore are forced to lag behind the most recent developments.In this paper, we make a step towards resolving these issues by proposing a general communication-efficient distribu...

Abstract:In this paper, we explore the potential of extreme learning machine (ELM) and kernel ELM (KELM) for early diagnosis of Parkinson's disease (PD). In the proposed method, the key parameters including the number of hidden neuron and type of activation function in ELM, and the constant parameter C and kernel parameter γ in KELM are investigated in detail. With the obtained optimal parameters, ELM and KELM manage to train the optimal predictive models for PD diagnosis. In order to further improve the performance of ELM and KELM models, feature selection techniques are implemented prior to the construction of the classification models. The effectiveness of the proposed method has been rigorously evaluated against the PD data set in terms of classification accuracy, sensitivity, specificity and the area under the ROC (receiver operating characteristic) curve (AUC). Compared to the existing methods in previous studies, the proposed method has achieved very promising classification accuracy via 10-fold cross-validation (CV) analysis, with the highest accuracy of 96.47% and average accuracy of 95.97% over 10 runs of 10-fold CV.

No abstract

No abstract

No abstract