Fast learning neural networks using Cartesian genetic programming

Khan, Maryam Mahsal; Ahmad, Arbab Masood; Khan, Gul Muhammad; Miller, Julian F.

doi:10.1016/j.neucom.2013.04.005

Cited by 82 publications

(12 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Koza and Rice [1991] brought a new dimension in this area, by using genetic evolution to optimize both, the weight and structure of a neural network. There have been many other works in this area [Palmes, Hayasaka, and Usui, 2005, Rivero, Dorado, Rabuñal, and Pazos, 2010, Tsai and Lin, 2011, Turner, Dudek, and Ritchie, 2010, Mahsal Khan, Masood Ahmad, Muhammad Khan, and Miller, 2013 however; none of the modifications is capable of delivering satisfactory performance for all problems, in general. Thus, the search for an approach to speed up its convergence and find the optimal structure still remain important.…”

Section: Artificial Neural Networkmentioning

confidence: 99%

Breast cancer diagnosis using Genetically Optimized Neural Network model

Bhardwaj

Tiwari

2015

Expert Systems with Applications

216

View full text Add to dashboard Cite

Section: Artificial Neural Networkmentioning

confidence: 99%

Breast cancer diagnosis using Genetically Optimized Neural Network model

Bhardwaj

Tiwari

2015

Expert Systems with Applications

216

View full text Add to dashboard Cite

“…To train SLFN (single hidden layer feedforward neural network) and to optimize the weight of SLFN, an algorithm is proposed by that hybridize the selforganizing map (SOM) algorithm with ELM algorithm [30]. CGP-based Artificial Neural Network (CGPANN), based on the Cartesian genetic programming (CGP) technique, is a fast-learning neuroevolutionary algorithm applicable for both feedforward and recurrent networks [31]. Even though the following methods produce good outcomes, they are computationally intensive.…”

Section: Related Studymentioning

confidence: 99%

Multi-Disease Classification Model Using Strassen’s Half of Threshold (SHoT) Training Algorithm in Healthcare Sector

et al. 2021

View full text Add to dashboard Cite

In healthcare industry, Neural Network has attained a milestone in solving many real-life classification problems varies from very simple to complex and from linear to non-linear. To improve the training process by reducing the training time, Adaptive Skipping Training algorithm named as Half of Threshold (HOT) has been proposed. To perform the fast classification and also to improve the computational efficiency such as accuracy, error rate, etc., the highlighted characteristics of proposed HOT algorithm has been integrated with Strassen's matrix multiplication algorithm and derived a novel, hybrid and computationally efficient algorithm for training and validating the neural network named as Strassen's Half of Threshold (SHoT) Training Algorithm. The experimental outcome based on the simulation demonstrated that the proposed SHOT algorithm outperforms both BPN and HOT algorithm in terms of training time and its efficiency on various dataset such as such as Hepatitis, SPeCT, Heart, Liver Disorders, Breast Cancer Wisconsin (Diagnostic), Drug Consumption, Cardiotocography, Splice-junction Gene Sequences and Thyroid Disease dataset that are extracted from Machine Learning Dataset Repository of UCI. It can be integrated with any type of supervised training algorithm.

show abstract

“…Theoretically, due to universal approximation theorem [9], this is the most expressive activation function; however, it requires many more parameters. Some of the early attempts to learn activation functions in neural networks can be found in Poli [39], Weingaertner et al [50], and Khan et al [22], where the authors proposed learning the best activation function per neuron among a pool of candidate activation functions using genetic and evolutionary algorithms. Maxout [15] has been introduced as an activation function aimed at enhancing the model averaging properties of dropout [45].…”

Section: Related Workmentioning

confidence: 99%

SPLASH: Learnable Activation Functions for Improving Accuracy and Adversarial Robustness

Tavakoli¹,

Agostinelli²,

Baldi³

2020

Preprint

View full text Add to dashboard Cite

We introduce SPLASH units, a class of learnable activation functions shown to simultaneously improve the accuracy of deep neural networks while also improving their robustness to adversarial attacks. SPLASH units have both a simple parameterization and maintain the ability to approximate a wide range of non-linear functions. SPLASH units are: 1) continuous; 2) grounded (f (0) = 0); 3) use symmetric hinges; and 4) the locations of the hinges are derived directly from the data (i.e. no learning required). Compared to nine other learned and fixed activation functions, including ReLU and its variants, SPLASH units show superior performance across three datasets (MNIST, CIFAR-10, and CIFAR-100) and four architectures (LeNet5, . Furthermore, we show that SPLASH units significantly increase the robustness of deep neural networks to adversarial attacks. Our experiments on both black-box and open-box adversarial attacks show that commonly-used architectures, namely LeNet5, All-CNN, ResNet-20, and Network-in-Network, can be up to 31% more robust to adversarial attacks by simply using SPLASH units instead of ReLUs.

show abstract

Fast learning neural networks using Cartesian genetic programming

Cited by 82 publications

References 50 publications

Breast cancer diagnosis using Genetically Optimized Neural Network model

Breast cancer diagnosis using Genetically Optimized Neural Network model

Multi-Disease Classification Model Using Strassen’s Half of Threshold (SHoT) Training Algorithm in Healthcare Sector

SPLASH: Learnable Activation Functions for Improving Accuracy and Adversarial Robustness

Contact Info

Product

Resources

About