Parallel Architectures for Learning the RTRN and Elman Dynamic Neural Networks

Bilski, Jarosław; Smoląg, Jacek

doi:10.1109/tpds.2014.2357019

Cited by 45 publications

(12 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other directions of future works include investigation of further optimization strategies for parallizing RBM learning, as well as possibility to use the hybrid environment provided by CPUs and coprocessors. The promising approach to the learning problem which should be considered is using various types of computational intelligence systems [2,11,12,13,16,17,18,19,20,27,41,43,44] including neural networks, in particular convolutional neural networks [28]. Such systems and their parallel implementation on Intel MIC architecture can be useful in many applications, such as face recognition [33,34,35,36].…”

Section: Discussionmentioning

confidence: 99%

Adaptation of RBM Learning for Intel MIC Architecture

Olas

Mleczko

Nowicki

et al. 2015

Artificial Intelligence and Soft Computing

View full text Add to dashboard Cite

Abstract. In the paper, the parallel realization of the Boltzmann Restricted Machine (RBM) is proposed. The implementation intends to use multicore architectures of modern CPUs and Intel Xeon Phi coprocessor. The learning procedure is based on the matrix description of RBM, where the learning samples are grouped into packages, and represented as matrices. The influence of the package size on convergence of learning, as well as on performance of computation, are studied for various number of threads, using conventional CPU and Intel Phi architecures. Our research confirms a potential usefulness of MIC parallel architecture for implementation of RBM and similar algorithms.

show abstract

Section: Discussionmentioning

confidence: 99%

Adaptation of RBM Learning for Intel MIC Architecture

Olas

Mleczko

Nowicki

et al. 2015

Artificial Intelligence and Soft Computing

View full text Add to dashboard Cite

show abstract

“…To efficiently handle large-scale CNN and big data, outstanding distributed architecture based solutions were implemented in [14,20]. Adam [21] is an efficient and scalable deep learning training system, optimizing and balancing workload computation and communication through entire system co-design.…”

Section: Related Workmentioning

confidence: 99%

A Bi-layered Parallel Training Architecture for Large-Scale Convolutional Neural Networks

Chen¹,

Li²,

Bilal

et al. 2019

IEEE Trans. Parallel Distrib. Syst.

162

View full text Add to dashboard Cite

Benefitting from large-scale training datasets and the complex training network, Convolutional Neural Networks (CNNs) are widely applied in various fields with high accuracy. However, the training process of CNNs is very time-consuming, where large amounts of training samples and iterative operations are required to obtain high-quality weight parameters. In this paper, we focus on the time-consuming training process of large-scale CNNs and propose a Bi-layered Parallel Training (BPT-CNN) architecture in distributed computing environments. BPT-CNN consists of two main components: (a) an outer-layer parallel training for multiple CNN subnetworks on separate data subsets, and (b) an inner-layer parallel training for each subnetwork. In the outer-layer parallelism, we address critical issues of distributed and parallel computing, including data communication, synchronization, and workload balance. A heterogeneousaware Incremental Data Partitioning and Allocation (IDPA) strategy is proposed, where large-scale training datasets are partitioned and allocated to the computing nodes in batches according to their computing power. To minimize the synchronization waiting during the global weight update process, an Asynchronous Global Weight Update (AGWU) strategy is proposed. In the inner-layer parallelism, we further accelerate the training process for each CNN subnetwork on each computer, where computation steps of convolutional layer and the local weight training are parallelized based on task-parallelism. We introduce task decomposition and scheduling strategies with the objectives of thread-level load balancing and minimum waiting time for critical paths. Extensive experimental results indicate that the proposed BPT-CNN effectively improves the training performance of CNNs while maintaining the accuracy.

show abstract

“…The first group of parameters of the proposed system are the parameters describing differences between the reference signatures and the templates in the partitions. They are used in the construction of fuzzy rules described later (see (9)) and determined as follows: …”

Section: Training Phasementioning

confidence: 99%

A New Method for the Dynamic Signature Verification Based on the Stable Partitions of the Signature

Zalasiński

Cpałka

2015

Artificial Intelligence and Soft Computing

View full text Add to dashboard Cite

Abstract. Dynamic signature is a very interesting biometric attribute which is commonly socially acceptable. In this paper we propose a new method for the dynamic signature verification using stable partitions of the signature. This method assumes selection of two the most stable hybrid partitions individually for the signer. Hybrid partitions are formed by a combination of vertical and horizontal sections of the signature. The selected partitions are used during identity verification process. In the test of the proposed method we used BioSecure DS2 database, distributed by the BioSecure Association.

show abstract

Parallel Architectures for Learning the RTRN and Elman Dynamic Neural Networks

Cited by 45 publications

References 15 publications

Adaptation of RBM Learning for Intel MIC Architecture

Adaptation of RBM Learning for Intel MIC Architecture

A Bi-layered Parallel Training Architecture for Large-Scale Convolutional Neural Networks

A New Method for the Dynamic Signature Verification Based on the Stable Partitions of the Signature

Contact Info

Product

Resources

About