Parallel orthogonal deep neural network

Mashhadi, Peyman Sheikholharam; Nowaczyk, Sławomir; Pashami, Sepideh

doi:10.1016/j.neunet.2021.03.002

Cited by 11 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Commonly, PSO is to observe the ideal outcome of the population with the assistance of particles' individual value, fitness value, and global best value. Parallel computing is used to achieve the desired accuracy and is used to accelerate neural network training for large training data sets [17,18,19,20]. In [17], a parallelization strategy for convolutional neural network (CNN) training is proposed based on two major techniques to maximize the overlap.…”

Section: Introductionmentioning

confidence: 99%

“…In [19], a parallel algorithm called Split and Conquer for solving Verification of Neural Network (VNN) formulas, is proposed using the Reluplex procedure and an iterative-deepening strategy. In [20], a parallel deep neural network architecture with an embedded organization mechanism is proposed, which enforces diversity among the deep neural networks used as base models. The Parallel Particle Swarm Optimization (PPSO) algorithm is an example of a parallel computing process to minimize the processing time of high computation [21,22,23].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Intelligent Decision-Making of Load Balancing Using Deep Reinforcement Learning and Parallel PSO in Cloud Environment

Pradhan

Bisoy

Kautish³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Machine learning and parallel processing are extremely commonly used to enhance computing power to induce knowledge from an outsized volume of data. To deal with the problem of complexity and high dimension, machine learning algorithms like Deep Reinforcement Learning (DRL) are used, while parallel processing algorithms like Parallel Particle Swarm Optimization (PPSO) are parallelized to speed up the operation and reduce the processing time to train the neural network. Due to the arrival of a large number of incoming tasks in the cloud environment, load balancing is an important issue. To solve this problem, the datacenter controller or an agent makes an intelligent decision to handle a large number of tasks within a minimum time period or at high speed. In this work, we proposed an effective scheduling algorithm named Deep Reinforcement Learning with Parallel Particle Swarm Optimization (DRLPPSO) to solve the load balancing problem and its various parameters with greater accuracy and high speed. Our experimental results show that our proposed scheduling algorithm increases the reward by 15.7%, 12%, and 13.1% when the task set is 2000 and improves the reward by 17.5%, 12.6%, and 15.3% when the task set is 4000, as compared to the Modified Particle Swarm Optimization (MPSO), Asynchronous Advantage Actor-Critic (A3C), and Deep Q-Network (DQN) techniques.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Intelligent Decision-Making of Load Balancing Using Deep Reinforcement Learning and Parallel PSO in Cloud Environment

Pradhan

Bisoy

Kautish³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Benchmark datasets . The benchmark data sets used in this study include the Davis, [ 43 ] Metz, [ 44 ] KIBA, [ 45 ] and PDBbind [ 46 ] data sets. The binding affinities of the four data sets were measured in dissociation constant ( K d ), inhibition constant (K i ), KIBA metric, and dissociation constant ( K d ).…”

Section: Methodsmentioning

confidence: 99%

“…It was noted that the calculation of the KIBA metric is shown in Equation 1. [ 45 ] Therefore, when calculating the generalization, the results of the Pearson correlation coefficient ( Rp ) and the Spearman correlation coefficient ( Rs) tested between KIBA data set and other data sets were taken as the opposite number. The formula for the conversion of K d and binding free energy (

\Delta {G}_{{\mathrm{bind}}}

) is shown in.…”

Section: Methodsmentioning

confidence: 99%

Prediction of Protein‐Ligand Binding Affinity by a Hybrid Quantum‐Classical Deep Learning Algorithm

Dong

Li²,

Liu

et al. 2023

Adv Quantum Tech

View full text Add to dashboard Cite

Rapid and accurate prediction of protein‐ligand binding affinity plays a vital role in high‐throughput drug screening. With the development of deep learning, increasingly accurate prediction models have been established. Deep learning may have ushered in an era of quantization, but the practical use of this theory for protein‐ligand binding affinity is still infrequent. Here, the introduction of the quantum algorithm into classical deep learning is described, which enables it to reliably predict protein‐ligand binding affinity using simple sequence information. Based on different deep learning models, including graph neural networks (GNN) and convolutional neural networks (CNN), corresponding quantum hybrid deep learning models have been constructed and compared to the classical models. This study has shown that the quantum algorithm can achieve considerable accuracy and good generalization, and show potential to balance between accuracy and generalization, although the parameters used in the model have been remarkably reduced. These models based on quantum hybrid deep learning (QDL) show robust predictions on four benchmark datasets, and exhibit the practical application power in drug screening for targets related to human liver cirrhosis. This work highlights the potential of the hybrid quantum deep learning algorithm in solving complex problems in bioinformatics.

show abstract

“…r) i by varing α. In(Larrazabal et al 2021;Ayinde, Inanc, and Zurada 2019;Mashhadi, Nowaczyk, and Pashami 2021), inducing orthogonality (i.e., forcing cosine similarity to zero) between parameters prevents a model learning redundant features given learning capacity. In this context, we expect the local model of each client wl(r) i to learn client-specific knowledge that cannot be captured by training of the global model w g(r) i in PHASE I.…”

mentioning

confidence: 99%

Subspace Learning for Personalized Federated Optimization

Hahn,

Jeong,

Lee

2021

Preprint

View full text Add to dashboard Cite

As data is generated and stored almost everywhere, learning a model from a data-decentralized setting is a task of interest for many AI-driven service providers. Although federated learning is settled down as the main solution in such situations, there still exists room for improvement in terms of personalization. Training federated learning systems usually focuses on optimizing a global model that is identically deployed to all client devices. However, a single global model is not sufficient for each client to be personalized on their performance as local data assumes to be not identically distributed across clients. We propose a method to address this situation through the lens of ensemble learning based on the construction of a low-loss subspace continuum that generates a high-accuracy ensemble of two endpoints (i.e. global model and local model). We demonstrate that our method achieves consistent gains both in personalized and unseen client evaluation settings through extensive experiments on several standard benchmark datasets.

show abstract

Parallel orthogonal deep neural network

Cited by 11 publications

References 16 publications

Intelligent Decision-Making of Load Balancing Using Deep Reinforcement Learning and Parallel PSO in Cloud Environment

Intelligent Decision-Making of Load Balancing Using Deep Reinforcement Learning and Parallel PSO in Cloud Environment

Prediction of Protein‐Ligand Binding Affinity by a Hybrid Quantum‐Classical Deep Learning Algorithm

Subspace Learning for Personalized Federated Optimization

Contact Info

Product

Resources

About