Improvement of Generalization Ability of Deep CNN via Implicit Regularization in Two-Stage Training Process

Zheng, Qinghe; Yang, Mo; Yang, Jingzhou; Zhang, Qingrui; Zhang, Xinxin

doi:10.1109/access.2018.2810849

Cited by 231 publications

(129 citation statements)

References 35 publications

Supporting

Mentioning

128

Contrasting

Order By: Relevance

“…8 reveals that the location of some landmarks may appear occasionally more accurate in output of the presented methods than in the original so-called ground-truth. This is a well known effect of machine learning methods, and DL methods in particular, which tend to avoid encoding noise and outliers by means of regularization techniques 50,51 . In general, a larger dataset labelled by several experts may limit the impact of the uncertainties in the ground-truth data and reinforce the robustness the DL methods and their resistance to noise.…”

Section: Number Of Epochsmentioning

confidence: 99%

Automatic vocal tract landmark localization from midsagittal MRI data

Eslami

Neuschaefer‐Rube

Serrurier

2020

Sci Rep

View full text Add to dashboard Cite

the various speech sounds of a language are obtained by varying the shape and position of the articulators surrounding the vocal tract. Analyzing their variations is crucial for understanding speech production, diagnosing speech disorders and planning therapy. identifying key anatomical landmarks of these structures on medical images is a pre-requisite for any quantitative analysis and the rising amount of data generated in the field calls for an automatic solution. The challenge lies in the high inter-and intra-speaker variability, the mutual interaction between the articulators and the moderate quality of the images. This study addresses this issue for the first time and tackles it by means of Deep Learning. it proposes a dedicated network architecture named Flat-net and its performance are evaluated and compared with eleven state-of-the-art methods from the literature. the dataset contains midsagittal anatomical Magnetic Resonance Images for 9 speakers sustaining 62 articulations with 21 annotated anatomical landmarks per image. Results show that the Flat-net approach outperforms the former methods, leading to an overall Root Mean Square Error of 3.6 pixels/0.36 cm obtained in a leave-oneout procedure over the speakers. the implementation codes are also shared publicly on GitHub.www.nature.com/scientificreports www.nature.com/scientificreports/ animals 22 . This short review emphasizes the importance of anatomical landmark localization from images for a large variety of applications and this study lengthens this non-exhaustive list with speech production.Within the existing literature, two fields of application appear more particularly active and connected to our problem. The first is the localization of landmarks for the face, rich of an abundant research from landmark identification on two-dimensional photographs 23,24 to landmark identification on three-dimensional Ultrasounds for fetuses for instance 25,26 . A recent and comprehensive review is provided by Wu et al. 27 . It represents a challenging issue due to the high variability of the shapes, poses, occlusions, and lighting conditions. Similarly, localizing the position of the joints of the body on images to estimate the human pose is also a long-standing problem 28 . It is also a challenging issue in computer vision due to the high variability of the postures, body shapes, actions, clothes and scenes.The goal and contribution of this study is to propose a fully-automated end-to-end image analysis methods for localizing key anatomical landmarks in the vocal tract area from midsagittal MRI data. As emphasized earlier, the need for such a method in the field is required and has never been attempted so far. It is aimed to be used in the future for new speakers for which no prior data are available. Considering the existing methods of the literature and the recent rise of data science to solve such problems, Deep Learning (DL) appears as the inescapable approach 29 . Indeed, DL approaches in image processing tasks appear to outperform most of traditional t...

show abstract

Section: Number Of Epochsmentioning

confidence: 99%

Automatic vocal tract landmark localization from midsagittal MRI data

Eslami

Neuschaefer‐Rube

Serrurier

2020

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Sun et al [ 25 ] proposed SVD-Net to show that guaranteeing the feature weight of the FC layer can increase the orthogonal constraint of the network and improve the accuracy. Zheng et al [ 26 ] reported that regularization was an efficient method for improving the generalization ability of deep CNN because it makes it possible to train more complex models while maintaining lower overfitting. Zheng et al [ 26 ] proposed a method for optimizing the feature boundary of a deep CNN through a two-stage training step to reduce the overfitting problem.…”

Section: Related Workmentioning

confidence: 99%

“…Zheng et al [ 26 ] reported that regularization was an efficient method for improving the generalization ability of deep CNN because it makes it possible to train more complex models while maintaining lower overfitting. Zheng et al [ 26 ] proposed a method for optimizing the feature boundary of a deep CNN through a two-stage training step to reduce the overfitting problem. However, the mixed features learned from CNN potentially reduce the robustness of network models for identification or classification.…”

Section: Related Workmentioning

confidence: 99%

SVD-CNN: A Convolutional Neural Network Model with Orthogonal Constraints Based on SVD for Context-Aware Citation Recommendation

Tao

Shen

Zhu

et al. 2020

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Context-aware citation recommendation aims to automatically predict suitable citations for a given citation context, which is essentially helpful for researchers when writing scientific papers. In existing neural network-based approaches, overcorrelation in the weight matrix influences semantic similarity, which is a difficult problem to solve. In this paper, we propose a novel context-aware citation recommendation approach that can essentially improve the orthogonality of the weight matrix and explore more accurate citation patterns. We quantitatively show that the various reference patterns in the paper have interactional features that can significantly affect link prediction. We conduct experiments on the CiteSeer datasets. The results show that our model is superior to baseline models in all metrics.

show abstract

“…Although a previous study [13] showed that the hyper-parameters and CNN architecture affected the overall accuracy, the CNN architecture of GoogLeNet is suitable for performing classification in many fields owing to its high accuracy; thus, we used fixed parameters. Second, the process of training accuracy and loss were not showed in this study because the ability to generalize was most important for the intended application [18] in the training process; thus, we only focused on the overall accuracy. However, the overfitting would hardly cause problems during training in this study because GoogLeNet adopted the inception module [19] and global average pooling [20] for preventing overfitting.…”

Section: Automatic Trainingmentioning

confidence: 99%

Evaluating the Overall Accuracy of Additional Learning and Automatic Classification System for CT Images

Sugimori

2019

Applied Sciences

View full text Add to dashboard Cite

A large number of images that are usually registered images in a training dataset are required for creating classification models because training of images using a convolutional neural network is done using supervised learning. It takes a significant amount of time and effort to create a registered dataset because recently computed tomography (CT) and magnetic resonance imaging devices produce hundreds of images per examination. This study aims to evaluate the overall accuracy of the additional learning and automatic classification systems for CT images. The study involved 700 patients, who were subjected to contrast or non-contrast CT examination of brain, neck, chest, abdomen, or pelvis. The images were divided into 500 images per class. The 10-class dataset was prepared with 10 datasets including with 5000–50,000 images. The overall accuracy was calculated using a confusion matrix for evaluating the created models. The highest overall reference accuracy was 0.9033 when the model was trained with a dataset containing 50,000 images. The additional learning for manual training was effective when datasets with a large number of images were used. The additional learning for automatic training requires models with an inherent higher accuracy for the classification.

show abstract

Improvement of Generalization Ability of Deep CNN via Implicit Regularization in Two-Stage Training Process

Cited by 231 publications

References 35 publications

Automatic vocal tract landmark localization from midsagittal MRI data

Automatic vocal tract landmark localization from midsagittal MRI data

SVD-CNN: A Convolutional Neural Network Model with Orthogonal Constraints Based on SVD for Context-Aware Citation Recommendation

Evaluating the Overall Accuracy of Additional Learning and Automatic Classification System for CT Images

Contact Info

Product

Resources

About