Memory effects description by neural networks with delayed feedback connections

Abstract

For the purpose of dynamic systems modeling, it was proposed to include feedback connections or delay elements in the classical feed-forward neural network structure so that the present output of the neural network depends on its previous values. These delay elements can be connected to the hidden and/or output neurons of the main neural network. Each delay element gets a value of a state variable at a past time instant and keeps this value during a single sampling period. The groups of delay elements record the values of the state variables for a given time period in the past. Changing the number of the delay elements, which belongs to one group, a shorter or a longer time period in the past can be accounted for. Thus, the connection weights determine the influence of the past process states on the present state in a similar way as it is in the time delay kernel or cause-effect relation membership function (CER-MF) models. Specific feed-forward neural networks with time delay connections are used to solve the problem of neural network chemostat modeling as well as specific kinetic rates modeling. The weights of the feedback connections obtained during model training are discussed as the points of a time delay kernel or as the strength levels in a CER model (the points in the CER-MF). The corresponding changes in these weights with the changing time period in the past are shown.