Wavelets as activation functions in Neural Networks

Herrera, Oscar; Sánchez, Belém Priego

doi:10.3233/jifs-219225

Cited by 5 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the above function cannot be directly computed, such an activation function can be approximated by Equations ( 17) and (18).…”

Section: Gelu Functionmentioning

confidence: 99%

“…While these activation functions utilize nonlinear information to enhance the problem‐solving capabilities of linear models, they suffer from the vanishing gradient problem. [ 18 ] Subsequently, the Gaussian error linear unit (GELU) was proposed, demonstrating promising results in the Modified National Institute of Standards and Technology (MNIST) classifier. [ 19 ] GELU addresses the issue of gradient disappearance by incorporating part of the negative information when x < 0.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

GELU‐LSTM‐encoder‐decoder fault prediction for batch processes based on the global–local percentile method

Gao,

Li,

et al. 2024

Can J Chem Eng

View full text Add to dashboard Cite

Fault prediction is a crucial strategy for ensuring the safety and efficiency of batch processes, and it aims to predict the degradation trend and isolate the fault. Operating mistakes or external environment changes can lead to the degradation of batch processes, eventually resulting in faults. The point at which degradation begins is known as the time to start prediction (TSP). Determining the TSP provides more reliable degradation information for fault prediction and enhances prediction accuracy. Nevertheless, it is difficult to obtain TSP in batch processes because of some complex data features, such as multi‐phase and strong dynamics, which result in a significant loss of fault prediction. To address these challenges, a fault prediction method is proposed based on the global–local percentile method applied to Gaussian error linear unit‐long short‐term memory‐encoder‐decoder (GELU‐LSTM‐encoder‐ED). Firstly, squared prediction error (SPE) is normalized to eliminate differences in data features. Secondly, a global–local percentile method is proposed to determine the TSP. The period is divided based on the threshold fluctuation to derive the model input. Then the global percentile model is employed to identify the interval to which the TSP belongs. Subsequently, the local percentile model is employed to analyze this interval and identify the TSP. Thirdly, the LSTM based on the encoder‐decoder model, enhanced with the GELU, is utilized to improve the performance of fault prediction. Finally, two fault datasets on the penicillin fermentation process and Escherichia coli fermentation process are employed to demonstrate the performance of the presented method compared with other shared methods.

show abstract

“…Since the above function cannot be directly computed, such an activation function can be approximated by Equations ( 17) and (18).…”

Section: Gelu Functionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

GELU‐LSTM‐encoder‐decoder fault prediction for batch processes based on the global–local percentile method

Gao,

Li,

et al. 2024

Can J Chem Eng

View full text Add to dashboard Cite

show abstract

“…Besides, mathematical model analysis is one of the common mechanical analysis methods and method of electrical fault diagnosis and analysis for heavy-duty robots. Nowadays, major diagnosis methods of robot faults ,there are expert system diagnosis methods, artificial neural network methods and mathematical model-based methods [22][23][24]. The development of a new expert system for diagnosing marine diesel engines based on real-time diagnostic parameters proposed by Gharib [25] , and tEstimation of the Kinematics and Workspace of a Robot Using Artificial Neural Networks proposed by Boanta apply artificial neural network to evaluate the workspace of robots [26] .…”

Section: Introductionmentioning

confidence: 99%

Research on Fault pattern Recognition of High-speed and Heavy-load Robots based on Median Neural Network

Xie,

Zhao,

Zhu

2024

Preprint

View full text Add to dashboard Cite

To address difficult recognition of vibration signals of heavy-load robots, a new median neural network pattern recognition method based on artificial neural network was proposed. This method builds the pattern recognition network model by using the median neurons and particle swarm optimization algorithm. In other words, the fault modes of heavy-load industrial robots are recognized quickly and accurately by using median and multiplication function as the aggregate function, and combining with the particle swarm backpropagation algorithm under the premise that there’s abnormal or fault eigenvalues in the network input mode. According to the simulation analysis and case verification, the proposed median neural network method can recognize fault modes of heavy-load robots well and it still and a smaller one recognition fault rate more traditional algorithms In below the complicated severe environment with noises. It shows remarkable recognition accuracy as well as relatively high compatibility and stability. This method not only has high theoretical value in pattern recognition of heavy-duty robots, but also possesses very important engineering practice significance in fault pattern recognition of rotary machines like ventilator and gas blower.The proposed method not only has high theoretical value in pattern recognition of heavy-duty robots, but there are also important,it has practical significance in engineering and other fields fault pattern recognition of rotating machinery such as ventilators and gas blowers.

show abstract

“…At the same time, machine learning models are suitable for the accurate estimation of complex nonlinear relationships. Therefore, inspired by the success of this hybrid prediction model, an integrated model is proposed, which is based on singular spectrum analysis (SSA), ARIMA, − Prophet, and wavelet neural network (WNN), , for predicting daily oil production. The proposed method consists of the following main steps.…”

Section: Introductionmentioning

confidence: 99%

Research on the Application of Integrated Learning Models in Oilfield Production Forecasting

Ni,

Xin,

et al. 2023

ACS Omega

View full text Add to dashboard Cite

Forecasting oil production is crucially important in oilfield management. Currently, multifeature-based modeling methods are widely used, but such modeling methods are not universally applicable due to the different actual conditions of oilfields in different places. In this paper, a time series forecasting method based on an integrated learning model is proposed, which combines the advantages of linearity and nonlinearity and is only concerned with the internal characteristics of the production curve itself, without considering other factors. The method includes processing the production history data using singular spectrum analysis, training the autoregressive integrated moving average model and Prophet, training the wavelet neural network, and forecasting oil production. The method is validated using historical production data from the J oilfield in China from 2011 to 2021, and compared with single models, Arps model, and mainstream time series forecasting models. The results show that in the early prediction, the difference in prediction error between the integrated learning model and other models is not obvious, but in the late prediction, the integrated model still predicts stably and the other models compared with it will show more obvious fluctuations. Therefore, the model in this article can make stable and accurate predictions.

show abstract

Wavelets as activation functions in Neural Networks

Cited by 5 publications

References 10 publications

GELU‐LSTM‐encoder‐decoder fault prediction for batch processes based on the global–local percentile method

GELU‐LSTM‐encoder‐decoder fault prediction for batch processes based on the global–local percentile method

Research on Fault pattern Recognition of High-speed and Heavy-load Robots based on Median Neural Network

Research on the Application of Integrated Learning Models in Oilfield Production Forecasting

Contact Info

Product

Resources

About