A survey of methods for distributed machine learning

Abstract

Traditionally, a bottleneck preventing the development of more intelligent systems was the limited amount of data available. Nowadays, the total amount of information is almost incalculable and automatic data analyzers are even more needed. However, the limiting factor is the inability of learning algorithms to use all the data to learn within a reasonable time. In order to handle this problem, a new field in machine learning has emerged: large-scale learning. In this context, distributed learning seems to be a promising line of research since allocating the learning process among several workstations is a natural way of scaling up learning algorithms. Moreover, it allows to deal with data sets that are naturally distributed, a frequent situation in many real applications. This study provides some background regarding the advantages of distributed environments as well as an overview of distributed learning for dealing with "very large" data sets.

A survey of machine learning for big data processing

“…However, a bottleneck preventing such a big blessing is the inability of learning algorithms to use all the data to learn within a reasonable time. In this context, distributed learning seems to be a promising research since allocating the learning process among several workstations is a natural way of scaling up learning algorithms [42]. Different from the classical learning framework, in which one requires the collection of that data in a database for central processing, in the framework of distributed learning, the learning is carried out in a distributed manner [43].…”

“…With the advantage of distributed computing for managing big volumes of data, distributed learning avoids the necessity of gathering data into a single workstation for central processing, saving time and energy. It is expected that more widespread applications of the distributed learning are on the way [42]. Similar to distributed learning, another popular learning technique for scaling up traditional learning algorithms is parallel machine learning [48].…”

“…Similar to distributed learning, another popular learning technique for scaling up traditional learning algorithms is parallel machine learning [48]. With the power of multicore processors and cloud computing platforms, parallel and distributed computing systems have recently become widely accessible [42]. A more detailed description about distributed and parallel learning can be found in [49].…”

Modelling of the relation of implication with use of the directed relational networks

“…В работе [13] дан обзор методов распределенного машинного обучения. Для автоматического машинного анализа больших объемов данных предлагается использовать метод крупномасштабного обучения.…”

Neuromorphic Computing with Memristor Crossbar

“…Since ANNs and SNNs simulate different characteristics of the biological neural networks, ANNs based deep neural networks (DNNs) and SNNs based learning systems are both being developed continuously for different purposes . Currently, implementations of neural networks are mostly based on the von Neumann computing system (VCS) such as CPU, GPU, and their cluster, which is powerful for logical computing but not efficient for neuronal and synaptic computing . Figure (a) illustrates the complexity relationship between the data environment and machine.…”