Parallel implementation of multilayered neural networks based on Map-Reduce on cloud computing clusters

Abstract

To meet the requirements of big data processing, this paper presents an efficient mapping scheme for a fully connected multilayered neural network, which is trained by using back-propagation (BP) algorithm based on MapReduce of cloud computing clusters. The batch-training (or epoch-training) regimes are used by effective segmentation of samples on the clusters, and are adopted in the separated training method, weight summary to achieve convergence by iterating. For a parallel BP algorithm on the clusters and a serial BP algorithm on an uniprocessor, the required time for implementing the algorithms is derived. The performance parameters, such as speedup, optimal number and minimum of data nodes are evaluated for the parallel BP algorithm on the clusters. Experiment results demonstrate that the proposed parallel BP algorithm in this paper has better speedup, faster convergence rate, less iterations than that of the existed algorithms.

Hardware Acceleration on Cloud Services: The use of Restricted Boltzmann Machines on Handwritten Digits Recognition

“…The proposed architecture is based on equations (11), (12) and (14) and is organized into three separate phases, shown in Figure 5. The time required to execute each phase determines the performance increase that can be achieved by proper pipeline.…”

“…Accelerator Architecture used for the calculations. The proposed design uses fixed-point arithmetic for all linear functions (add, mul, cmp) and single-precision floating-point for non-linear functions, like log and exp in(12). To specify the range of the fixed-point arithmetic, simulations were run and statistics were collected based on all available training and test data patterns.…”

Scalable machine‐learning algorithms for big data analytics: a comprehensive review

“…It was found that among all, MapReduce performs the worst, with a slight improvement in the case of Haloop, and Spark performs the best due to in‐memory computation. Similarly, Zhang and Xiao have implemented a parallel multilayered neural network employing a BP training algorithm on MapReduce (MRBP) on cloud computing clusters for speedup.…”

Semantics Interaction Control for Constructing Intelligent Ecology of Internet of Things and Critical Component Research

“…This has always been one of the ultimate goals of artificial intelligence, which is one of critical components to structure the intelligent interconnections of the Internet of Things. Intelligent voice interaction technology has involuntarily become one of the current research hotspots [4] [5] [6] [7] [8].…”