No abstract

The hyper-parameters of a neural network are traditionally designed through a time-consuming process of trial and error that requires substantial expert knowledge. Neural Architecture Search algorithms aim to take the human out of the loop by automatically finding a good set of hyper-parameters for the problem at hand. These algorithms have mostly focused on hyper-parameters such as the architectural configurations of the hidden layers and the connectivity of the hidden neurons, but there has been relatively little work on automating the search for completely new activation functions, which are one of the most crucial hyperparameters to choose. There are some widely used activation functions nowadays that are simple and work well, but nonetheless, there has been some interest in finding better activation functions. The work in the literature has mostly focused on designing new activation functions by hand or choosing from a set of predefined functions while this work presents an evolutionary algorithm to automate the search for completely new activation functions. We compare these new evolved activation functions to other existing and commonly used activation functions. The results are favorable and are obtained from averaging the performance of the activation functions found over 30 runs, with experiments being conducted on 10 different datasets and architectures to ensure the statistical robustness of the study.

Agriculture remains an important sector of the economy. Plant diseases and pests have a big impact on plant yield and quality. So, prevention and early detection of crop disease are some of the measures that must be implemented in farming to save the plants at an early stage and thereby reduce the overall food loss. Grapes are the most profitable fruit, but they are also vulnerable to a variety of diseases. Black Measles, Black Rot, and Leaf Blight are diseases that affect grape plants. Manual disease diagnosis can result in improper identification and use of pesticides, and it takes a long time. A variety of deep learning approaches have been used to address this issue of the identification and classification of grape leaf diseases. However, there are also limits to such approaches. Therefore, this paper uses deep learning with the concept of ensemble learning based on three famous Convolutional Neural Network (CNN) architectures (Visual Geometry Group (VGG16), VGG19, and Extreme Inception (Xception)). These three models are pretrained with ImageNet. The performance of the proposed approach is analyzed using the Plant Village (PV) dataset of common grape leaf diseases. The Proposed model gives higher performance than the results achieved by using each Deep Learning architecture separately and compared with the recent approaches in this study. The proposed system outperformed the others with 99.82% accuracy.

No abstract

The introduction of the ReLU function in neural network architectures yielded substantial improvements over sigmoidal activation functions and allowed for the training of deep networks. Ever since, the search for new activation functions in neural networks has been an active research topic. However, to the best of our knowledge, the design of new activation functions has mostly been done by hand. In this work, we propose the use of a self-adaptive evolutionary algorithm that searches for new activation functions using a genetic programming approach, and we compare the performance of the obtained activation functions to ReLU. We also analyze the shape of the obtained activations to see if they have any common traits such as monotonicity or piece-wise linearity, and we study the effects of the self-adaptation to see which operators perform well in the context of a search for new activation functions. We perform a thorough experimental study on datasets of different sizes and types, using different types of neural network architectures. We report favorable results obtained from the mean and standard deviation of the performance metrics over multiple runs.