Visualising basins of attraction for the cross-entropy and the squared error neural network loss functions

Bosman, Anna Sergeevna; Engelbrecht, Andries P.; Helbig, Mardé

doi:10.1016/j.neucom.2020.02.113

Cited by 62 publications

(47 citation statements)

References 33 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To minimize the overfitting, the process of the training was forced to finish before repetition K if no improvement was perceived for seven iterations, this control was made using early stopping [50] . As the COVID-19 dataset used is a binary-class classification problem, the DenseNet121 is compiled with the binary cross-entropy [51] . The Adam optimizer algorithm [52] was used with a constant learning rate =2e-5 within the feature extraction method.…”

Section: Experiments Resultsmentioning

confidence: 99%

An optimized deep learning architecture for the diagnosis of COVID-19 disease based on gravitational search optimization

Ezzat

Hassanien

Ella

2021

Applied Soft Computing

158

View full text Add to dashboard Cite

In this paper, a novel approach called GSA-DenseNet121-COVID-19 based on a hybrid convolutional neural network (CNN) architecture is proposed using an optimization algorithm. The CNN architecture that was used is called DenseNet121, and the optimization algorithm that was used is called the gravitational search algorithm (GSA). The GSA is used to determine the best values for the hyperparameters of the DenseNet121 architecture. To help this architecture to achieve a high level of accuracy in diagnosing COVID-19 through chest x-ray images. The obtained results showed that the proposed approach could classify 98.38% of the test set correctly. To test the efficacy of the GSA in setting the optimum values for the hyperparameters of DenseNet121. The GSA was compared to another approach called SSD-DenseNet121, which depends on the DenseNet121 and the optimization algorithm called social ski driver (SSD). The comparison results demonstrated the efficacy of the proposed GSA-DenseNet121-COVID-19. As it was able to diagnose COVID-19 better than SSD-DenseNet121 as the second was able to diagnose only 94% of the test set. The proposed approach was also compared to another method based on a CNN architecture called Inception-v3 and manual search to quantify hyperparameter values. The comparison results showed that the GSA-DenseNet121-COVID-19 was able to beat the comparison method, as the second was able to classify only 95% of the test set samples. The proposed GSA-DenseNet121-COVID-19 was also compared with some related work. The comparison results showed that GSA-DenseNet121-COVID-19 is very competitive.

show abstract

Section: Experiments Resultsmentioning

confidence: 99%

An optimized deep learning architecture for the diagnosis of COVID-19 disease based on gravitational search optimization

Ezzat

Hassanien

Ella

2021

Applied Soft Computing

158

View full text Add to dashboard Cite

show abstract

“…Bosman et al [16][17][18][19] applied and adapted standard fitness landscape analysis techniques to error landscapes. Studies include: the influence of search space boundaries on the landscape analysis [16], the influence of regularisation on error surfaces [17], the influence of architecture settings on modality of the landscape [18], and the effect of different loss functions on the basins of attraction [19].…”

Section: Error Landscapesmentioning

confidence: 99%

A Survey of Advances in Landscape Analysis for Optimisation

Malan

2021

Algorithms

View full text Add to dashboard Cite

Fitness landscapes were proposed in 1932 as an abstract notion for understanding biological evolution and were later used to explain evolutionary algorithm behaviour. The last ten years has seen the field of fitness landscape analysis develop from a largely theoretical idea in evolutionary computation to a practical tool applied in optimisation in general and more recently in machine learning. With this widened scope, new types of landscapes have emerged such as multiobjective landscapes, violation landscapes, dynamic and coupled landscapes and error landscapes. This survey is a follow-up from a 2013 survey on fitness landscapes and includes an additional 11 landscape analysis techniques. The paper also includes a survey on the applications of landscape analysis for understanding complex problems and explaining algorithm behaviour, as well as algorithm performance prediction and automated algorithm configuration and selection. The extensive use of landscape analysis in a broad range of areas highlights the wide applicability of the techniques and the paper discusses some opportunities for further research in this growing field.

show abstract

“…In general, each algorithm has a loss function in deep learning tasks [37], [38]. The algorithm minimizes loss function to allow better model fitting training data.…”

Section: ) Focus On Weightsmentioning

confidence: 99%

Color Space Transformation and Multi-Class Weighted Loss for Adhesive White Blood Cell Segmentation

Zhao

et al. 2020

IEEE Access

View full text Add to dashboard Cite

White blood cells (WBCs) are the cells of immune system, protecting against infective diseases and invasion of viruses and bacteria. Their aberrant number, both abnormal increase and decrease, is a sign of an ongoing pathology, a precise evaluation of their number is of the utmost importance as the first step of assessing a potential disease. In blood cell microscopic images, since red blood cells and platelets are similar in color with WBCs, and WBCs are partially adhesive, WBC segmentation for counting is often not resulting in a good performance. Therefore, in this work, a color space transformation is proposed to filter out red blood cells and platelets, which is transforming the blood cell microscopic images of patients with acute lymphoblastic leukemia from RGB color space to HSV to detect and extract WBCs. For precisely segmenting adhesive WBCs in extraction results, we set cell border to the third class, in addition to foreground and background. A weighted cross-entropy loss function based on class weight and distance transformation weight enhanced U-Net to learn cell border features. Our results showed that the method proposed in this paper for WBC segmentation using the data set ALL_IDB1 could achieve an accuracy of 97.92%.

show abstract

Visualising basins of attraction for the cross-entropy and the squared error neural network loss functions

Cited by 62 publications

References 33 publications

An optimized deep learning architecture for the diagnosis of COVID-19 disease based on gravitational search optimization

An optimized deep learning architecture for the diagnosis of COVID-19 disease based on gravitational search optimization

A Survey of Advances in Landscape Analysis for Optimisation

Color Space Transformation and Multi-Class Weighted Loss for Adhesive White Blood Cell Segmentation

Contact Info

Product

Resources

About