RSigELU: A nonlinear activation function for deep neural networks

Kılıçarslan, Serhat; Çelik, Mete

doi:10.1016/j.eswa.2021.114805

Cited by 73 publications

(32 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, the ReLU activation function is faces the problem of negative region. Since negative values are set to zero, the derivative of the values cannot be taken and the learning process slows down [4,18]. One of the biggest advantages of the ReLU activation function is that the computational load is low compared to other functions and it can be widely used in multi-layered architectures.…”

Section: Relu Activation Functionmentioning

confidence: 99%

“…Among the non-linear functions, the parameterized activation function is the functions that allow us to work actively on deep learning architectures thanks to the parameter value. Parametric activation functions used in the literature are expressed as LReLU, ELU, SELU, RSigELU [4,[8][9][10][11][12].…”

Section: Parametric Activation Functionsmentioning

confidence: 99%

“…Today, deep learning architectures are actively preferred by researchers in many areas such as autonomous vehicles, image processing, signal processing, and prediction [1][2][3]. Deep learning architectures are consisted of such as the layers convolution, max-pooling, activation, dropout, normalization, flatten, full connection and softmax [4]. Activation functions have a significant contribution to the development of activation functions by scientists due to their positive contribution to learning on deep neural networks [5].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An overview of the activation functions used in deep learning algorithms

Kılıçarslan

Adem

Çelik

2021

Journal of New Results in Science

Self Cite

View full text Add to dashboard Cite

In deep learning models, the inputs to the network are processed using activation functions to generate the output corresponding to these inputs. Deep learning models are of particular importance in analyzing big data with numerous parameters and forecasting and are useful for image processing, natural language processing, object recognition, and financial forecasting. Also, in deep learning algorithms, activation functions have been developed by taking into account features such as performing the learning process in a healthy way, preventing excessive learning, increasing the accuracy performance, and reducing the computational cost. In this study, we present an overview of common and current activation functions used in deep learning algorithms. In the study, fixed and trainable activation functions are introduced. As fixed activation functions, sigmoid, hyperbolic tangent, ReLU, softplus and swish, and as trainable activation functions, LReLU, ELU, SELU and RSigELU are introduced.

show abstract

Section: Relu Activation Functionmentioning

confidence: 99%

Section: Parametric Activation Functionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An overview of the activation functions used in deep learning algorithms

Kılıçarslan

Adem

Çelik

2021

Journal of New Results in Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the formula, g j v,c represents the weight of neuron j in the connection layer; I j v represents the offset of neuron j; and N represents the number of neurons. Input the value of neuron j into a nonlinear activation function z [18]:…”

Section: Feature Recognition Of Artistic Visual Imagementioning

confidence: 99%

“…Based on the sampling law, increase the spatial resolution of the artistic visual image according to twice the range [20], and the original viewpoint value will be affected and changed due to the increase of high-frequency component. At this time, use formula (18) to shift the control point:…”

Section: Defolding Of Artistic Visual Imagesmentioning

confidence: 99%

Art Visual Image Transmission Method Based on Cartesian Genetic Programming

Zhao

2021

Scientific Programming

View full text Add to dashboard Cite

Because most of the traditional artistic visual image communication methods use the form of modeling and calculation, there are some problems such as long image processing time, low success rate of image visual communication, and poor visual effect. An artistic visual image communication method based on Cartesian genetic programming is proposed. The visual expression sensitivity difference method is introduced to process the image data, the neural network is used to identify the characteristics of the artistic visual image, the midpoint displacement method is used to remove the folds of the artistic visual image, and the processed image is formed under the above three links. The Cartesian genetic programming algorithm is used to encode the preprocessed image, improve the fitness function, select the algorithm to improve the operation, design the image rendering platform, input the processed image to the platform, and complete the artistic visual image transmission. The analysis of the experimental results shows that the image processing time of this method is short, the success rate of visual communication is high, and the image visual effect is good, which can obtain the image processing results satisfactory to users.

show abstract

High‐performance optical noncontact controlling system based on broadband PtTe_x/Si heterojunction photodetectors for human–machine interaction

Ran

Yan

et al. 2021

InfoMat

View full text Add to dashboard Cite

Noncontact interaction systems have attracted considerable research attention in recent years because of convenient operation, sterility, and injury prevention. However, the insufficient sensing distance and weak robustness of noncontact interaction systems for complex environments limit their practical applications. Here, we designed an integrated optical noncontact controlling system (ONCS) based on PtTex/Si optoelectronic heterojunction array. Broadband sensitive photoresponse is realized at zero bias voltage, with excellent detectivity and responsivity, boosting the noncontact sensing distance to at least 150 mm. Consequently, the system can perform noncontact detection, encoding, and control by recognizing shadow‐induced spatiotemporal sequence changes in heterojunction array photocurrents. As a proof of concept, different interactive functions have been demonstrated with good accuracy and robustness by encoding finger movement above the ONCS. This study provides a new perspective for constructing high‐performance noncontact interaction systems.

show abstract

RSigELU: A nonlinear activation function for deep neural networks

Cited by 73 publications

References 18 publications

An overview of the activation functions used in deep learning algorithms

An overview of the activation functions used in deep learning algorithms

Art Visual Image Transmission Method Based on Cartesian Genetic Programming

High‐performance optical noncontact controlling system based on broadband PtTe_x/Si heterojunction photodetectors for human–machine interaction

Contact Info

Product

Resources

About

RSigELU: A nonlinear activation function for deep neural networks

Cited by 73 publications

References 18 publications

An overview of the activation functions used in deep learning algorithms

An overview of the activation functions used in deep learning algorithms

Art Visual Image Transmission Method Based on Cartesian Genetic Programming

High‐performance optical noncontact controlling system based on broadband PtTex/Si heterojunction photodetectors for human–machine interaction

Contact Info

Product

Resources

About

High‐performance optical noncontact controlling system based on broadband PtTe_x/Si heterojunction photodetectors for human–machine interaction