Improving deep neural networks with multi-layer maxout networks and a novel initialization method

Sun, Weichen; Su, Fei; Wang, Leiquan

doi:10.1016/j.neucom.2017.05.103

Cited by 102 publications

(57 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sun et al proposed Multi-layer Maxout Networks (MMN) with multi-layer which can train active function, and deduced a new initialization method dedicated to the activation of MMN. The method can reduce the movement of internal covariates when the signal propagates through the layer [ 6 ]. A. Pacheco determined the input weights and bias for the Extreme Learning Machine (ELM) by using the Restricted Boltzmann Machine (RBM), named as RBM-ELM [ 4 ].…”

Section: Introductionmentioning

confidence: 99%

PCA Based Kernel Initialization for Convolutional Neural Networks

Wang

Rong

Liu

et al. 2020

Communications in Computer and Information Science

View full text Add to dashboard Cite

The initialization of Convolutional Neural Networks (CNNs) is about providing reasonable initial values for the convolution kernels and the fully connected layers. In this paper, we proposed a convolution kernel initialization method based on the two-dimensional principal component analysis (2DPCA), in which a parametric equalization normalization method is used to adjust the scale between each neuron weight. After that the weight initial value can be adaptively adjusted according to different data samples. This method enables each neuron to fully back-propagate errors and accelerate network model training. Finally, a network model was built and experiments were performed using Tanh and ReLU activation functions. The experimental results verify the effectiveness of the proposed method through the distribution of histograms and the curve comparison diagrams of model training.

show abstract

Section: Introductionmentioning

confidence: 99%

PCA Based Kernel Initialization for Convolutional Neural Networks

Wang

Rong

Liu

et al. 2020

Communications in Computer and Information Science

View full text Add to dashboard Cite

show abstract

“…After extracting the features from three modalities, the feature fusion process is carried out using the proposed EDHO algorithm, which is derived by the integration of EHO 27 and DHOA, 28 respectively. Finally, the process of recognition is carried out using Deep Maxout network 29 . Figure 1 portrays the schematic view of the proposed EDHO‐based hybrid fusion model.…”

Section: Proposed Elephant Deer Hunting Optimization‐based Hybrid Fusion Modelmentioning

confidence: 99%

A nature‐inspired meta‐heuristic paradigm for person identification using multimodal biometrics

Mohan¹,

Ganesan²

2021

Concurrency and Computation

View full text Add to dashboard Cite

Because of the expanding improvement of the internet, security toward individual data has gotten more significant in recent decades. Due to the increasing development of the internet, security toward personal information has become more important in recent decades. Various person recognition methods are introduced for ensuring the security of a person's information. However identifying the information of the user using unique physiological characteristics poses a major issue in the biometric recognition system. Hence, an effective multimodal recognition system is developed using the proposed Elephant Deer Hunting Optimization‐based hybrid fusion (EDHO‐based hybrid fusion) model for person recognition. The proposed EDHO is designed by the integration of Elephant Herding Optimization and Deer Hunting Optimization Algorithm. Here, three different modalities, such as finger vein, dorsal hand vein, and electromyography (EMG) data are employed for the person recognition process. To increase the level of security, the hybrid feature fusion process is employed in such a way that features are fused based on the weight coefficients. The optimization algorithm is considered for the computation of the weight factor. However, the optimal weight value shows the optimal solution of person recognition in such a way that the computation of optimal value is based on the fitness measure. The proposed method achieved higher performance of accuracy as 0.963, True Negative Rate (TNR) as 0.9658, and True Positive rate (TPR) as 0.9548, respectively.

show abstract

“…Two interesting and successful approaches are Maxout [Goodfellow et al, 2013, Sun et al, 2018 and NIN [Lin et al, 2013]. However, despite the good performances, both approaches move away from the concept of trainable activation function as it has been previously discussed insofar as the adaptable function does not correspond to the neuron activation function by which the neuron output is computed on the basis of a scalar value (the neuron input) according to the standard two-stage process.…”

Section: Other Approachesmentioning

confidence: 99%

A simple and efficient architecture for trainable activation functions

Apicella¹,

Isgrò²,

Prevete³

2019

Neurocomputing

View full text Add to dashboard Cite

Learning automatically the best activation function for the task is an active topic in neural network research. At the moment, despite promising results, it is still difficult to determine a method for learning an activation function that is at the same time theoretically simple and easy to implement. Moreover, most of the methods proposed so far introduce new parameters or adopt different learning techniques. In this work we propose a simple method to obtain trained activation function which adds to the neural network local subnetworks with a small amount of neurons. Experiments show that this approach could lead to better result with respect to using a pre-defined activation function, without introducing a large amount of extra parameters that need to be learned.

show abstract

Improving deep neural networks with multi-layer maxout networks and a novel initialization method

Cited by 102 publications

References 10 publications

PCA Based Kernel Initialization for Convolutional Neural Networks

PCA Based Kernel Initialization for Convolutional Neural Networks

A nature‐inspired meta‐heuristic paradigm for person identification using multimodal biometrics

A simple and efficient architecture for trainable activation functions

Contact Info

Product

Resources

About