Design and analysis of recurrent neural network models with non‐linear activation functions for solving time‐varying quadratic programming problems

This study proposes a novel gradient‐based neural network model with an activated variable parameter, named as the activated variable parameter gradient‐based neural network (AVPGNN) model, to solve time‐varying constrained quadratic programming (TVCQP) problems. Compared with the existing models, the AVPGNN model has the following advantages: (1) avoids the matrix inverse, which can significantly reduce the computing complexity; (2) introduces the time‐derivative of the time‐varying parameters in the TVCQP problem by adding an activated variable parameter, enabling the AVPGNN model to achieve a predictive calculation that achieves zero residual error in theory; (3) adopts the activation function to accelerate the convergence rate. To solve the TVCQP problem with the AVPGNN model, the TVCQP problem is transformed into a non‐linear equation with a non‐linear compensation problem function based on the Karush Kuhn Tucker conditions. Then, a variable parameter with an activation function is employed to design the AVPGNN model. The accuracy and convergence rate of the AVPGNN model are rigorously analysed in theory. Furthermore, numerical experiments are also executed to demonstrate the effectiveness and superiority of the proposed model. Moreover, to explore the feasibility of the AVPGNN model, applications to the motion planning of a robotic manipulator and the portfolio selection of marketed securities are illustrated.

Section: Robot Motion Planningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An activated variable parameter gradient‐based neural network for time‐variant constrained quadratic programming and its applications

Wang

Hao

et al. 2023

“…The emergence of neural dialogue systems solves the problems of mechanical rigidity and narrow application fields of traditional dialogue systems. A recurrent neural network (RNN) framework based on sequence-to-sequence (Seq2Seq) has been successfully applied to dialogue systems [3][4][5][6][7][8]. These dialogue systems mainly focus on the understanding of semantics and the improvement of the generated responses.…”

Section: Introductionmentioning

confidence: 99%

A semantic and emotion‐based dual latent variable generation model for a dialogue system

Yan

Lou

Chan

et al. 2023

With the development of intelligent agents pursuing humanisation, artificial intelligence must consider emotion, the most basic spiritual need in human interaction. Traditional emotional dialogue systems usually use an external emotional dictionary to select appropriate emotional words to add to the response or concatenate emotional tags and semantic features in the decoding step to generate appropriate responses. However, selecting emotional words from a fixed emotional dictionary may result in loss of the diversity and consistency of the response. We propose a semantic and emotion-based dual latent variable generation model (Dual-LVG) for dialogue systems, which is able to generate appropriate emotional responses without an emotional dictionary. Different from previous work, the conditional variational autoencoder (CVAE) adopts the standard transformer structure. Then, Dual-LVG regularises the CVAE latent space by introducing a dual latent space of semantics and emotion. The content diversity and emotional accuracy of the generated responses are improved by learning emotion and semantic features respectively. Moreover, the average attention mechanism is adopted to better extract semantic features at the sequence level, and the semi-supervised attention mechanism is used in the decoding step to strengthen the fusion of emotional features of the model. Experimental results show that Dual-LVG can successfully achieve the effect of generating different content by controlling emotional factors.

“…Zeroing neural network (ZNN) is a special RNN model created by Zhang et al. [13–15], which is designed for handling time‐varying problem by using the derivative information of time‐varying parameter. In the above study, researchers have proposed and studied lots of models for solving time‐varying matrix inversion.…”

Section: Introductionmentioning

confidence: 99%

An advanced discrete‐time RNN for handling discrete time‐varying matrix inversion: Form model design to disturbance‐suppression analysis

Shi

Cao

et al. 2023

Time‐varying matrix inversion is an important field of matrix research, and lots of research achievements have been obtained. In the process of solving time‐varying matrix inversion, disturbances inevitably exist, thus, a model that can suppress disturbance while solving the problem is required. In this paper, an advanced continuous‐time recurrent neural network (RNN) model based on a double integral RNN design formula is proposed for solving continuous time‐varying matrix inversion, which has incomparable disturbance‐suppression property. For digital hardware applications, the corresponding advanced discrete‐time RNN model is proposed based on the discretisation formulas. As a result of theoretical analysis, it is demonstrated that the advanced continuous‐time RNN model and the corresponding advanced discrete‐time RNN model have global and exponential convergence performance, and they are excellent for suppressing different disturbances. Finally, inspiring experiments, including two numerical experiments and a practical experiment, are presented to demonstrate the effectiveness and superiority of the advanced discrete‐time RNN model for solving discrete time‐varying matrix inversion with disturbance‐suppression.