A tight upper bound on the generalization error of feedforward neural networks

Sarraf, Aydin

doi:10.1016/j.neunet.2020.04.001

Cited by 5 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BPNN [31], DNN [32], DBM and DBM-GA. The proposed method is tested against various performance metrics like accuracy, specificity, sensitivity, F-measure, execution time and mean error rate.…”

Section: Resultsmentioning

confidence: 99%

Iot Device Classification in Cloud Organizations Using Deep Belief Networks

M²,

et al. 2021

Preprint

View full text Add to dashboard Cite

With the rapid increase in the usage of IoT devices, the cyber threats are increasing among the communication between the IoT devices. The challenges related to security surmounts with increasing number of IoT devices due to its functionality and heterogeneity. In recent times, deep learning algorithms are offered to resolve the constraints associated with detection of malicious devices among the networks. In this paper, we utilize deep belief network (DBN) to resolve the problems associated with identification, detection of anomaly IoT devices. Several features are extracted initially to find the malicious devices in the IoT device network that includes storage, computational resources and high dimensional features. These features extracted from the network traffic assists in achieving the classification of devices by DBN. The simulation is performed to test the accuracy and detection rate of the proposed deep learning classifier. The results show that the proposed method is effective in implementing the detection of malicious nodes in the network than existing methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Iot Device Classification in Cloud Organizations Using Deep Belief Networks

M²,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…It can be seen from Figure 4 that the proposed RBFNN-GP has a good prediction performance for the permeability of the MBR, and the prediction error within the range [−4, 3]. Moreover, Table 3 shows the comparison results of SASOA-FNN [12], SOFNN-HPS [17], SAS-RBFNN [18], AGMOPSO [21], ASOL-SORBFNN [23] and Fixed-RBFNN in predicting the permeability of MBR. It can be seen from Table 3 that, under the same iteration times, the learning time of SASOA-FNN [12] and RBFNN-GP is almost the same.…”

Section: Permeability Prediction Of Membrane Bio-reactormentioning

confidence: 96%

“…It is clear that the proposed RBFNN-GP can approximate the Mexican straw hat function with small predicting errors. In order to further prove the excellent generalization ability of the proposed method, the prediction results of the RBFNN-GP are compared with those of the other dynamic neural networks based on structural adjustment, such as SASOA-FNN [12], SOFNN-HPS [17], SAS-RBFNN [18], AG-MOPSO [21], ASOL-SORBFNN [23] and the RBFNN with a fixed structure (fixed-RBFNN). In order to make the comparison more meaningful, all algorithms in this experiment use the same data set, including training samples and test samples, and ensure that the initial number of neurons is the same.…”

Section: Benchmark Example Amentioning

confidence: 99%

“…Similarly, Table 2 the comparative experimental results with the other four methods: SASOA-FNN [12], SOFNN-HPS [17], SAS-RBFNN [18], AGMOPSO [21], ASOL-SORBFNN [23] and fixed-RBFNN. As shown in Table 2, the optimal number of hidden nodes obtained by using the RBFNN-GP is only six, and the mean and deviation of the test error are minimal in the comparison method.…”

Section: Benchmark Example Bmentioning

confidence: 99%

“…By predefining the neighborhood of training samples in the local generalization error model, the upper bound of generalization error of invisible samples was derived. Sarraf [12] proposed a tight upper bound of the generalization error under the assumption of twice continuously differentiable, which was composed of the estimation error under the sample space mean and the expected sensitivity of the error to the input change, and showed how the given upper bound could be used to analyze the generalization error of a feedforward neural network. Although the above methods achieved good results through the generalization error bound based on sensitivity, they still faced challenges due to the computational complexity of partial derivatives, and the generalization error should not only be a function of the number of parameters; it is also important to find a better structure.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Novel Radial Basis Function Neural Network with High Generalization Performance for Nonlinear Process Modelling

2022

View full text Add to dashboard Cite

A radial basis function neural network (RBFNN), with a strong function approximation ability, was proven to be an effective tool for nonlinear process modeling. However, in many instances, the sample set is limited and the model evaluation error is fixed, which makes it very difficult to construct an optimal network structure to ensure the generalization ability of the established nonlinear process model. To solve this problem, a novel RBFNN with a high generation performance (RBFNN-GP), is proposed in this paper. The proposed RBFNN-GP consists of three contributions. First, a local generalization error bound, introducing the sample mean and variance, is developed to acquire a small error bound to reduce the range of error. Second, the self-organizing structure method, based on a generalization error bound and network sensitivity, is established to obtain a suitable number of neurons to improve the generalization ability. Third, the convergence of this proposed RBFNN-GP is proved theoretically in the case of structure fixation and structure adjustment. Finally, the performance of the proposed RBFNN-GP is compared with some popular algorithms, using two numerical simulations and a practical application. The comparison results verified the effectiveness of RBFNN-GP.

show abstract