Rates of approximation by ReLU shallow neural networks

Mao, Tong; Zhou, Ding-Xuan

doi:10.1016/j.jco.2023.101784

Cited by 10 publications

(1 citation statement)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding activation functions of the hidden neurons, defaults provided by the TensorFlow library were used. That is, for dense networks, ReLU [44] was used, and for RNNs and LSTMs hyperbolic tangent [45] was applied. For activation function of the output layer, ReLU was chosen, because the output of the network (V RMS−Radial ) is always non-negative.…”

Section: Neural Network Architectures and Training Processmentioning

confidence: 99%

Vibration Velocity Prediction with Regression and Forecasting Techniques for Axial Piston Pump

Fic,

Czornik,

Rosikowski

2023

Applied Sciences

View full text Add to dashboard Cite

Measuring vibration velocity is one of the most common techniques to estimate the condition of industrial machines. At a constant operating point, as the vibration velocity value increases, the machine’s condition worsens. However, there are no precise thresholds that indicate the condition of a machine at different operating points. Also, the axial piston pump, which is the subject of the article, is a device that generates stronger vibrations by design and cannot be enclosed in general vibration norms. Due to different use cases and work regimes of axial piston pumps, the need to determine whether the device is working correctly for a broad spectra of operating points emerges. This article aims to present and compare different methods for vibration velocity prediction for axial piston pumps with use of neural networks including dense networks, variants of recurrent neural networks, and ensemble methods. The result of this research consists of models that have performance metrics that clearly indicate whether the monitored pump has malfunctioned or not across a wide variety of operating points, working conditions, and in case of reassembling. A detailed analysis of the influence of available measured variables on the performance of models is also provided. The conclusion is that the application of commercial implementation of developed models is reasonable in the context of both performance quality and costs of sensors needed to provide the necessary data.

show abstract

Section: Neural Network Architectures and Training Processmentioning

confidence: 99%

Vibration Velocity Prediction with Regression and Forecasting Techniques for Axial Piston Pump

Fic,

Czornik,

Rosikowski

2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

Extracting Sentiment from Business News Announcements for More Efficient Decision Making

Liagkouras,

Metaxiotis

2024

Learning and Analytics in Intelligent Systems

View full text Add to dashboard Cite

Optimal Rates of Approximation by Shallow ReLU$$^k$$ Neural Networks and Applications to Nonparametric Regression

Yang,

Zhou

2024

Constr Approx

View full text Add to dashboard Cite

We study the approximation capacity of some variation spaces corresponding to shallow ReLU$$^k$$ k neural networks. It is shown that sufficiently smooth functions are contained in these spaces with finite variation norms. For functions with less smoothness, the approximation rates in terms of the variation norm are established. Using these results, we are able to prove the optimal approximation rates in terms of the number of neurons for shallow ReLU$$^k$$ k neural networks. It is also shown how these results can be used to derive approximation bounds for deep neural networks and convolutional neural networks (CNNs). As applications, we study convergence rates for nonparametric regression using three ReLU neural network models: shallow neural network, over-parameterized neural network, and CNN. In particular, we show that shallow neural networks can achieve the minimax optimal rates for learning Hölder functions, which complements recent results for deep neural networks. It is also proven that over-parameterized (deep or shallow) neural networks can achieve nearly optimal rates for nonparametric regression.

show abstract

Rates of approximation by ReLU shallow neural networks

Cited by 10 publications

References 38 publications

Vibration Velocity Prediction with Regression and Forecasting Techniques for Axial Piston Pump

Vibration Velocity Prediction with Regression and Forecasting Techniques for Axial Piston Pump

Extracting Sentiment from Business News Announcements for More Efficient Decision Making

Optimal Rates of Approximation by Shallow ReLU$$^k$$ Neural Networks and Applications to Nonparametric Regression

Contact Info

Product

Resources

About