An automatic selection of optimal recurrent neural network architecture for processes dynamics modelling purposes

Laddach, Krzysztof; Łangowski, Rafał; Puchalski, Bartosz

doi:10.1016/j.asoc.2021.108375

Cited by 14 publications

(3 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where: X i (t) is the input value at time t W ij (t) is the weight of neural input at time t b ij is the bias F is a transfer function y(t) is the output value at time t Recurrent neural networks (RNNs) are variants of neural networks that are good at dealing with sequential data processing [47]. The structure of the ANN is organized iteratively, such that output data are converted to input data taking into account the stored output of the previous time step t − 1, which is added to the inputs of the current time step t. This configuration means that a change in the state of an individual neuron can be transferred via feedback to the other neurons, invoking transient states and generally leading to another state of the network [48].…”

Section: Basic Concepts Of the Algorithms Used In This Studymentioning

confidence: 99%

Comparative Analysis of the Implementation of Support Vector Machines and Long Short-Term Memory Artificial Neural Networks in Municipal Solid Waste Management Models in Megacities

Meza

Yepes

Rodrigo-Ilarri

et al. 2023

IJERPH

View full text Add to dashboard Cite

The development of methodologies to support decision-making in municipal solid waste (MSW) management processes is of great interest for municipal administrations. Artificial intelligence (AI) techniques provide multiple tools for designing algorithms to objectively analyze data while creating highly precise models. Support vector machines and neuronal networks are formed by AI applications offering optimization solutions at different managing stages. In this paper, an implementation and comparison of the results obtained by two AI methods on a solid waste management problem is shown. Support vector machine (SVM) and long short-term memory (LSTM) network techniques have been used. The implementation of LSTM took into account different configurations, temporal filtering and annual calculations of solid waste collection periods. Results show that the SVM method properly fits selected data and yields consistent regression curves, even with very limited training data, leading to more accurate results than those obtained by the LSTM method.

show abstract

Section: Basic Concepts Of the Algorithms Used In This Studymentioning

confidence: 99%

Comparative Analysis of the Implementation of Support Vector Machines and Long Short-Term Memory Artificial Neural Networks in Municipal Solid Waste Management Models in Megacities

Meza

Yepes

Rodrigo-Ilarri

et al. 2023

IJERPH

View full text Add to dashboard Cite

show abstract

“…Figure 1 shows an illustrative example of the main concepts of NNs [19][20][21][22]. There are three layers in this example.…”

Section: Neural Optimization Machine (A) a Brief Introduction To Nnsmentioning

confidence: 99%

Neural optimization machine: a neural network approach for optimization and its application in additive manufacturing with physics-guided learning

Chen,

Liu

2023

Phil. Trans. R. Soc. A.

View full text Add to dashboard Cite

Neural networks (NNs) are increasingly used in design to construct the objective functions and constraints, which leads to the needs of optimization of NN models with respect to design variables. A Neural Optimization Machine (NOM) is proposed for constrained single/multi-objective optimization by appropriately designing the NN architecture, activation function and loss function. The NN's built-in backpropagation algorithm conducts the optimization and is seamlessly integrated with the additive manufacturing (AM) process-property model. The NOM is tested using several numerical optimization problems. It is shown that the increase in the dimension of design variables does not increase the computational cost significantly. Next, a brief review of the physics-guided machine learning model for fatigue performance prediction of AM components is given. Finally, the NOM is applied to design processing parameters in AM to optimize the mechanical fatigue properties through the physics-guided NN under uncertainties. One novel contribution of the proposed methodology is that the constrained process optimization is integrated with physics/knowledge and the data-driven AM process-property model. Thus, a physics-compatible process design can be achieved. Another significant benefit is that the training and optimization are achieved in a unified NN model, and no separate process optimization is needed. This article is part of the theme issue 'Physics-informed machine learning and its structural integrity applications (Part 1)'.

show abstract

“…At the initial stage, the neuron receives output signals at its input, which then summarize and exceed the activation function of each of them, resulting in the output signal (Figure 1a). There is a multilayer frontier neural network of the most direct propagation [19] (Figure 1b), which is an oriented graph with one-way directed edges. A significant disadvantage of neural networks that one may consider is the solving tasks aimed at black box-type pattern recognition.…”

Section: Introductionmentioning

confidence: 99%

Development of an Algorithm for Multicriteria Optimization of Deep Learning Neural Networks

Alexandrov

Kirichek²,

Kuklin

et al. 2023

HighTech. Innov. J.

View full text Add to dashboard Cite

Nowadays, machine learning methods are actively used to process big data. A promising direction is neural networks, in which structure optimization occurs on the principles of self-configuration. Genetic algorithms are applied to solve this nontrivial problem. Most multicriteria evolutionary algorithms use a procedure known as non-dominant sorting to rank decisions. However, the efficiency of procedures for adding points and updating rank values in non-dominated sorting (incremental non-dominated sorting) remains low. In this regard, this research improves the performance of these algorithms, including the condition of an asynchronous calculation of the fitness of individuals. The relevance of the research is determined by the fact that although many scholars and specialists have studied the self-tuning of neural networks, they have not yet proposed a comprehensive solution to this problem. In particular, algorithms for efficient non-dominated sorting under conditions of incremental and asynchronous updates when using evolutionary methods of multicriteria optimization have not been fully developed to date. To achieve this goal, a hybrid co-evolutionary algorithm was developed that significantly outperforms all algorithms included in it, including error-back propagation and genetic algorithms that operate separately. The novelty of the obtained results lies in the fact that the developed algorithms have minimal asymptotic complexity. The practical value of the developed algorithms is associated with the fact that they make it possible to solve applied problems of increased complexity in a practically acceptable time. Doi: 10.28991/HIJ-2023-04-01-011 Full Text: PDF

show abstract

An automatic selection of optimal recurrent neural network architecture for processes dynamics modelling purposes

Cited by 14 publications

References 36 publications

Comparative Analysis of the Implementation of Support Vector Machines and Long Short-Term Memory Artificial Neural Networks in Municipal Solid Waste Management Models in Megacities

Comparative Analysis of the Implementation of Support Vector Machines and Long Short-Term Memory Artificial Neural Networks in Municipal Solid Waste Management Models in Megacities

Neural optimization machine: a neural network approach for optimization and its application in additive manufacturing with physics-guided learning

Development of an Algorithm for Multicriteria Optimization of Deep Learning Neural Networks

Contact Info

Product

Resources

About