M-Estimators based activation functions for robust neural network learning

Essai, Mohamed H.; Ellah, Ali R. Abd

doi:10.1109/icenco.2014.7050434

Cited by 7 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dataset is then contaminated in the x-y axis by Gaussian noise with a mean of zero and a standard deviation of 0.1, G2~N (0, 0.1). A variable percentage, ε, of data was randomly selected and then replaced with probability, ε, by background noise uniformly distributed in the specific range [24,26,29,31,32].…”

Section: Simulation Resultsmentioning

confidence: 99%

“…The research [23] presented a set of robust, statistical M-estimators as a replacement for the conventional MSE loss function using good, noise-free information. Furthermore, new transfer functions that depend on robust statistical m-estimators proposed alternatives to traditional transfer functions using a dataset containing outliers [24].…”

Section: Relevant Workmentioning

confidence: 99%

“…When gross errors or outliers contaminate the data, it leads to an unreliable method. To overcome the problem of outliers, several robust statistical estimators that are not severely affected by deviations from observations, such as M-estimators [20,21,23,24,26,29,31], R-estimators, L-estimators, and LTS [30], and LMedS [37], have been used in recent research papers. In this paper, we address the robust training of DNN based on using M-estimators as robust loss functions to substitute the conventional MSE loss function, which loses robustness in the presence of outliers in the V2X dataset.…”

Section: Robust Learning and Outliersmentioning

confidence: 99%

See 2 more Smart Citations

Performance Estimation in V2X Networks Using Deep Learning-Based M-Estimator Loss Functions in the Presence of Outliers

et al. 2021

View full text Add to dashboard Cite

Recently, 5G networks have emerged as a new technology that can control the advancement of telecommunication networks and transportation systems. Furthermore, 5G networks provide better network performance while reducing network traffic and complexity compared to current networks. Machine-learning techniques (ML) will help symmetric IoT applications become a significant new data source in the future. Symmetry is a widely studied pattern in various research areas, especially in wireless network traffic. The study of symmetric and asymmetric faults and outliers (anomalies) in network traffic is an important topic. Nowadays, deep learning (DL) is an advanced approach in challenging wireless networks such as network management and optimization, anomaly detection, predictive analysis, lifetime value prediction, etc. However, its performance depends on the efficiency of training samples. DL is designed to work with large datasets and uses complex algorithms to train the model. The occurrence of outliers in the raw data reduces the reliability of the training models. In this paper, the performance of Vehicle-to-Everything (V2X) traffic was estimated using the DL algorithm. A set of robust statistical estimators, called M-estimators, have been proposed as robust loss functions as an alternative to the traditional MSE loss function, to improve the training process and robustize DL in the presence of outliers. We demonstrate their robustness in the presence of outliers on V2X traffic datasets.

show abstract

Section: Simulation Resultsmentioning

confidence: 99%

Section: Relevant Workmentioning

confidence: 99%

Section: Robust Learning and Outliersmentioning

confidence: 99%

See 1 more Smart Citation

Performance Estimation in V2X Networks Using Deep Learning-Based M-Estimator Loss Functions in the Presence of Outliers

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Gomes et al [32] analyzed the performance of different activation functions in NN to accurately forecast time series data. Later, Essai and Ellah [33] performed experiments using robust M-estimators objective functions as activation functions which outperformed the activation functions used earlier in the literature. Freire and Barreto [34] used the idea of batch intrinsic plasticity (BIP) to maximize hidden layer information combined with robust estimation of the output weights.…”

Section: Introductionmentioning

confidence: 99%

Robust Extreme Learning Machine Using New Activation and Loss Functions Based on M-Estimation for Regression and Classification

Khan

Ali

Islam

et al. 2022

Scientific Programming

View full text Add to dashboard Cite

This paper provides an analysis of the combining effect of novel activation function and loss function based on M-estimation in application to extreme learning machine (ELM), a feed-forward neural network. Due to the computational efficiency and classification/prediction accuracy of ELM and its variants, they have been widely exploited in the development of new technologies and applications. However, in real applications, the performance of classical ELMs deteriorates in the presence of outliers, thus, negatively impacting the precision and accuracy of the system. To further enhance the performance of ELM and its variants, we proposed novel activation functions based on the psi function of M and redescend the M-estimation method along with the smooth l 2-norm weight-loss functions to reduce the negative impact of the outliers. The proposed psi functions of several M and redescending M-estimation methods are more flexible to make more distinct features space. For the first time, the idea of the psi function as an activation function in the neural network is introduced in the literature to ensure accurate prediction. In addition, new robust l 2 norm-loss functions based on M and redescending M-estimation are proposed to deal with outliers efficiently in ELM. To evaluate the performance of the proposed methodology against other state-of-the-art techniques, experiments have been performed in diverse environments, which show promising improvements in application to regression and classification problems.

show abstract