Petuum: A New Platform for Distributed Machine Learning on Big Data

Abstract

What is a systematic way to efficiently apply a wide spectrum of advanced ML programs to industrial scale problems, using Big Models (up to 100s of billions of parameters) on Big Data (up to terabytes or petabytes)? Modern parallelization strategies employ fine-grained operations and scheduling beyond the classic bulk-synchronous processing paradigm popularized by MapReduce, or even specialized graph-based execution that relies on graph representations of ML programs. The variety of approaches tends to pull systems and algorithms design in different directions, and it remains difficult to find a universal platform applicable to a wide range of ML programs at scale. We propose a general-purpose framework that systematically addresses data-and model-parallel challenges in large-scale ML, by observing that many ML programs are fundamentally optimization-centric and admit error-tolerant, iterative-convergent algorithmic solutions. This presents unique opportunities for an integrative system design, such as bounded-error network synchronization and dynamic scheduling based on ML program structure. We demonstrate the efficacy of these system designs versus well-known implementations of modern ML algorithms, allowing ML programs to run in much less time and at considerably larger model sizes, even on modestly-sized compute clusters.

Next-Generation Big Data Analytics: State of the Art, Challenges, and Future Research Topics

“…The IoTs is applied to assess whether circuits are associated and whether the equipment is running naturally to improve visiting efficien y, which ensures the safety of multimedia files Many researchers have attempted to produce systematic ways of applying a wide spectrum of advanced machine learning (ML) programs to industrial-scale problems. Xing et al proposed a general-purpose framework, Petuum, which systematically addresses data-and model-parallel challenges in large-scale ML by observing that many ML programs are fundamentally optimization-centric and admit error-tolerant, iterative-convergent algorithmic solutions [46]. This presents unique opportunities for an integrative system design, such as bounded-error network synchronization and dynamic scheduling based on an ML program structure.…”

Distributed Feature Selection for Efficient Economic Big Data Analysis

“…1. Specially, to speed up the process of data analysis, the Economic Data P reprocess and Economic F eature Selection are deployed in distributed platform [36].…”

An Efficient Industrial Big-Data Engine

“…Big-data offers many opportunities to evaluate data for example to identify preferences from end-users, to target fraud and also to relate other issues derived from a combined and statistical processing of data [3][5] [33]. Industrial big-data refers to large amounts of time series data generated at high-speed by industrial equipment, typically located in worldwide factories [5][6] [7][8] [9].…”