An Analysis of the Regularization Between L2 and Dropout in Single Hidden Layer Neural Network

Phaisangittisagul, Ekachai

doi:10.1109/isms.2016.14

Cited by 81 publications

(37 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, the detailed architecture of each component is elaborated. We, respectively, add the dropout layer [52] in DTB, UTB and bottleneck. Overfitting is a possibility as in deep convolutional neural network.…”

Section: Proposed Methodsmentioning

confidence: 99%

See 1 more Smart Citation

DenseUNet: densely connected UNet for electron microscopy image segmentation

et al. 2020

View full text Add to dashboard Cite

Section: Proposed Methodsmentioning

confidence: 99%

“…Overfitting is a possibility as in deep convolutional neural network. The dropout layer [52] is used to prevent the overfitting problem and improve the segmentation accuracy. In addition to the DB, the bottleneck also can reduce the parameter size in DenseUNet without segmentation performance decline.…”

Section: Proposed Methodsmentioning

confidence: 99%

DenseUNet: densely connected UNet for electron microscopy image segmentation

et al. 2020

View full text Add to dashboard Cite

“…Neurons are forced to learn, thus enhancing the network's generalization capability. L2 regularization improves generalization in linear models by penalizing weights in proportion to the sum of squares of weights [32]. We observed no signs of overfitting in the training graphs for the networks.…”

Section: Discussionmentioning

confidence: 67%

Detection of hand osteoarthritis from hand radiographs using convolutional neural networks with transfer learning

Üreten¹,

Erbay²,

Maraş³

2020

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

Osteoarthritis is the most common type of arthritis. Hand osteoarthritis leads to specific structural changes in the joints, such as asymmetric joint space narrowing and osteophytes (bone spurs). Conventional radiography has traditionally been the primary method of visualizing these structural changes and diagnosing osteoarthritis. We aimed to develop a computerized method that is capable of determining the structural changes seen in radiography of the hand and to assist practitioners in interpreting radiographic changes and diagnosing the disease. In this retrospective study, transfer-learning-based convolutional neural networks were trained on a randomly selected dataset containing 332 radiography images of hands from an original set of 420 and were validated with the remaining 88. Multilayer convolutional neural network models were designed based on a transfer learning method using pretrained AlexNet, GoogLeNet, and VGG-19 networks. The accuracies of the models were 93.2% for AlexNet, 94.3% for GoogLeNet, and 96.6% for VGG-19. The sensitivities of these models were 0.9167 for AlexNet, 0.9184 for GoogLeNet, and 0.9574 for VGG-19, while the specificity values were 0.9500, 0.9744, and 0.9756, respectively. The performance metrics, including accuracy, sensitivity, specificity, and precision, of our newly developed automated diagnosis methods are promising in the diagnosis of hand osteoarthritis. Our computer-aided detection systems may help physicians in interpreting hand radiography images, diagnosing osteoarthritis, and saving time.

show abstract

“…The number of neurons in each hidden layer of MLPs is heuristically set according to the problem to be processed. The more neurons there are, the stronger approximation ability of the network, and the greater risk of overfitting [ 20 ]. However, the number of neurons in the output layer is determined by the dimensions of the problem being processed.…”

Section: Principles Of the Basic Methodsmentioning

confidence: 99%

A Hybrid Method Applied to Improve the Efficiency of Full-Waveform Inversion for Pavement Characterization

Zhang

Yi³

et al. 2018

Sensors

View full text Add to dashboard Cite

Ground penetrating radar (GPR), as a nondestructive testing tool, is suitable for estimating the thickness and permittivity of layers within the pavement. However, it would become problematic when the layer is thin with respect to the probing pulse width, in which case overlapping between the reflected pulses occurs. In order to deal with this problem, a hybrid method based on multilayer perceptrons (MLPs) and a local optimization algorithm is proposed. This method can be divided into two stages. In the first stage, the MLPs roughly estimate the thickness and the permittivity of the GPR signal. In the second stage, these roughly estimated values are used as the initial solution of the full-waveform inversion algorithm. The hybrid method and the conventional global optimization algorithm are respectively used to perform the full-waveform inversion of the simulated GPR data. Under the same inversion precision, the objective function needs to be calculated for 450 times and 30 times for the conventional method and the hybrid method, respectively. The hybrid method is also applied to a measured data, and the thickness estimation error is 1.2 mm. The results show the high efficiency and accuracy of such hybrid method to resolve the problem of estimating the thickness and permittivity of a “thin layer”.

show abstract

An Analysis of the Regularization Between L2 and Dropout in Single Hidden Layer Neural Network

Cited by 81 publications

References 11 publications

DenseUNet: densely connected UNet for electron microscopy image segmentation

DenseUNet: densely connected UNet for electron microscopy image segmentation

Detection of hand osteoarthritis from hand radiographs using convolutional neural networks with transfer learning

A Hybrid Method Applied to Improve the Efficiency of Full-Waveform Inversion for Pavement Characterization

Contact Info

Product

Resources

About