Sine neural network (SNN) with double-stage weights and structure determination (DS-WASD)

Zhang, Yunong; Qu, Lu; Liu, Jinrong; Guo, Dongsheng; Li, Mingming

doi:10.1007/s00500-014-1491-6

Cited by 10 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, a weights and structure determination (WASD) method based on the weights direct determination (WDD) method is used to train the date. [28][29][30][31] Therefore WASD method is presented to analyze annual electricity production and a three-layer feedforward neuronet activated by sine is constructed in this paper, of which the model diagram is illustrated in Figure 1. The weights from the input layer to the hidden layer are determined randomly and the weights from the hidden layer to the output layer are calculated by the pseudo-inverse.…”

Section: Discussionmentioning

confidence: 99%

Forecasting of Chinese Hydropower Generation Using WASD-Neuronet

Cheng¹,

Ye²,

Guo³

et al. 2018

IJRC

View full text Add to dashboard Cite

Hydropower resource is one of the renewable energy sources. With the increasing Chinese economy, people are paying much more attention to sustainable development. The increasing hydropower load is the basis of the development of power industry. Due to the characteristics of electrical energy, predicting the hydropower accurately is a potentially beneficial way to plan hydropower reasonably. This paper presents a neural network method to predict hydropower generation whose data is influenced by several factors such as social economic, population and climate. By using the past 52-year rough data, a 3-layer feedforward neuronet equipped with the weights and structure determination (WASD) method is constructed for the prediction of the Chinese hydropower generation in this paper. By processing mass of data, we could basically predict the hydropower generation using such a WASD neuronet. To a large extent, the trend of developing Chinese hydropower generation in the next years will keep growing.

show abstract

Section: Discussionmentioning

confidence: 99%

Forecasting of Chinese Hydropower Generation Using WASD-Neuronet

Cheng¹,

Ye²,

Guo³

et al. 2018

IJRC

View full text Add to dashboard Cite

show abstract

“…The excitation function adopted by the traditional BP neural network is usually sigmoid, tanh, and ReLU, while this paper employs a set of linear independent orthogonal polynomials, which are Chebyshev polynomials instead of Sigmoid function, as the excitation function, as shown in Figure 5. A large amount of literature [17][18][19] has verified that using Chebyshev polynomials as the excitation function can effectively optimize the BP neural network. In the experiment of this paper, in contrast to networks and LSTM networks that use Sigmoid as the excitation function, the error of the network using CP as the excitation function declines faster and more steadily, and the prediction accuracy is also higher at the same time.…”

Section: Convert Sequence Features Into Target Output 231 Cp Combined...mentioning

confidence: 99%

“…There is a myriad of improvements in BP neural networks made by researchers, one of which is to change the excitation function of the BP neural network. For example, Zhang et al [17] took the sine function as the excitation function of the BP neural network. CP is a set of orthogonal polynomials that is often used for function approximation.…”

Section: Introductionmentioning

confidence: 99%

Research on Precipitation Forecast Based on LSTM–CP Combined Model

et al. 2021

View full text Add to dashboard Cite

The tremendous progress made in the field of deep learning allows us to accurately predict precipitation and avoid major and long-term disruptions to the entire socio-economic system caused by floods. This paper presents an LSTM–CP combined model formed by the Long Short-Term Memory (LSTM) network and Chebyshev polynomial (CP) as applied to the precipitation forecast of Yibin City. Firstly, the data are fed into the LSTM network to extract the time-series features. Then, the sequence features obtained are input into the BP (Back Propagation) neural network with CP as the excitation function. Finally, the prediction results are obtained. By theoretical analysis and experimental comparison, the LSTM–CP combined model proposed in this paper has fewer parameters, shorter running time, and relatively smaller prediction error than the LSTM network. Meanwhile, compared with the SVR model, ARIMA model, and MLP model, the prediction accuracy of the LSTM–CP combination model is significantly improved, which can aid relevant departments in making disaster response measures in advance to reduce disaster losses and promote sustainable development by providing them data support.

show abstract

“…It was, for example, used recently with a data-driven determination of a network's biases in [139]. Just the sine function without any scaling was used as an activation in [347][348][349][350][351][352].…”

Section: Sinementioning

confidence: 99%

On Transformative Adaptive Activation Functions in Neural Networks for Gene Expression Inference

Kunc

Kléma²

2019

Preprint

View full text Add to dashboard Cite

Motivation: Gene expression profiling was made cheaper by the NIH LINCS program that profiles only ∼1, 000 selected landmark genes and uses them to reconstruct the whole profile. The D-GEX method employs neural networks to infer the whole profile. However, the original D-GEX can be further significantly improved. Results: We have analyzed the D-GEX method and determined that the inference can be improved using a logistic sigmoid activation function instead of the hyperbolic tangent. Moreover, we propose a novel transformative adaptive activation function that improves the gene expression inference even further and which generalizes several existing adaptive activation functions. Our improved neural network achieves average mean absolute error of 0.1340 which is a significant improvement over our reimplementation of the original D-GEX which achieves average mean absolute error 0.1637

show abstract

Sine neural network (SNN) with double-stage weights and structure determination (DS-WASD)

Cited by 10 publications

References 17 publications

Forecasting of Chinese Hydropower Generation Using WASD-Neuronet

Forecasting of Chinese Hydropower Generation Using WASD-Neuronet

Research on Precipitation Forecast Based on LSTM–CP Combined Model

On Transformative Adaptive Activation Functions in Neural Networks for Gene Expression Inference

Contact Info

Product

Resources

About