Global dynamics of discrete neural networks allowing non-monotonic activation functions

Liz, Eduardo; Ruiz-Herrera, Alfonso

doi:10.1088/0951-7715/27/2/289

Cited by 5 publications

(3 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a recent paper [27], Zhuge, Sun, and Lei showed that the ω-limit set for any positive solution of (4.19) is contained in 17) which is exactly the set defined by the relation we derived in (4.16).…”

Section: μ(S)ds N(t A)mentioning

confidence: 85%

“…Equation (1.2) can describe the dynamics of a single neuron (or the mean field equation for a population of neurons in a network) with an excitatory and an inhibitory loop, having different delays. The pair of a positive and a negative delayed feedback occurs on the processing of sensory information and other processes in neuroscience (see [5,12,17,19] and references therein).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Global dynamics of delay recruitment models with maximized lifespan

El-Morshedy

Röst

Ruiz-Herrera

2016

Z. Angew. Math. Phys.

Self Cite

View full text Add to dashboard Cite

Abstract. We study the dynamics of the differential equationwith two delayed terms, representing a positive and a negative feedback. We prove delay-dependent and absolute global stability results for the trivial and for the positive equilibrium. Our theorems provide new mathematical results as well as novel insights for several biological systems, including three-stage populations, neural models with inhibitory and excitatory loops, and the blood platelet model of Bélair and Mackey. We show that, somewhat surprisingly, the introduction of a removal term with fixed delay in population models simplifies the dynamics of the equation.Mathematics Subject Classification. 39A30 · 39A33 · 37D45 · 92B20.

show abstract

Section: μ(S)ds N(t A)mentioning

confidence: 85%

Section: Introductionmentioning

confidence: 99%

Global dynamics of delay recruitment models with maximized lifespan

El-Morshedy

Röst

Ruiz-Herrera

2016

Z. Angew. Math. Phys.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The ELU is monotonic and approaches a constant value for negative inputs. Theoretically, the monotonic activation function should get better performance [19][20][21], ZHENG et al [22] proposed an improved activation function based on ELU and named FELU, which is also monotonic, while the Swish and Mish are not monotonic and both of them approach zero when increasing the negative inputs, some studies that tend to use non-monotonic activation function [23][24][25][26][27][28]. There are new proposed activation functions named RMAF [29] and REU (PREU) [30], which are similar to Swish but get better accuracy in their experiments and are also nonmonotonic.…”

Section: Introductionmentioning

confidence: 99%

PolyLU: A Simple and Robust Polynomial-Based Linear Unit Activation Function for Deep Learning

Feng,

Yang

2023

IEEE Access

View full text Add to dashboard Cite

The activation function has a critical influence on whether a convolutional neural network in deep learning can converge or not; a proper activation function not only makes the convolutional neural network converge faster but also can reduce the complexity of convolutional neural network architecture and gets the same or better performance. Many activation functions have been proposed; however, various activation functions have advantages, defects, and applicable network architectures. A new activation function called Polynomial Linear Unit (PolyLU) is proposed in this paper to improve some of the shortcomings of the existing activation functions. The PolyLU meets the following basic properties: continuously differentiable, approximate identity near the origin, unbounded for positive inputs, bounded for negative inputs, smooth, monotonic, and zero-centered. There is a polynomial term for the negative inputs and no exponential terms in the PolyLU that reduces the computational complexity of the network. Compared to those common activation functions like Sigmoid, Tanh, ReLU, LeakyReLU, ELU, Mish, and Swish, the experiments show that the PolyLU has improved some network complexity and has better accuracy over MNIST, Kaggle Cats and Dogs, CIFAR-10 and CIFAR-100 datasets. Test by the CIFAR-100 dataset with batch normalization, PolyLU improves by 0.

show abstract

Stability analysis of fractional-order neural networks: An LMI approach

Yang

Yong

et al. 2018

Neurocomputing

View full text Add to dashboard Cite

Global dynamics of discrete neural networks allowing non-monotonic activation functions

Cited by 5 publications

References 49 publications

Global dynamics of delay recruitment models with maximized lifespan

Global dynamics of delay recruitment models with maximized lifespan

PolyLU: A Simple and Robust Polynomial-Based Linear Unit Activation Function for Deep Learning

Stability analysis of fractional-order neural networks: An LMI approach

Contact Info

Product

Resources

About