Hyperparameter optimization of deep neural network using univariate dynamic encoding algorithm for searches

Yoo, Youngjun

doi:10.1016/j.knosys.2019.04.019

Cited by 107 publications

(45 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Hence, it is surprising that many DL applications in GP have not paid enough attention to this problem (Ma et al, 2017;Montesinos-López et al, 2018b;Montesinos-López et al, 2019b). Several approaches have been proposed for hyperparameter tuning (e.g., Bellot et al, 2018;Cho and Hegde, 2019;Le et al, 2019;Rajaraman et al, 2019;Yoo, 2019). Here, DL architectures were optimized using Talos (Autonomia Talos, 2019), which works combining all parameters in a grid.…”

Section: Hyperparameter Optimizationmentioning

confidence: 99%

Exploring Deep Learning for Complex Trait Genomic Prediction in Polyploid Outcrossing Species

et al. 2020

View full text Add to dashboard Cite

Section: Hyperparameter Optimizationmentioning

confidence: 99%

Exploring Deep Learning for Complex Trait Genomic Prediction in Polyploid Outcrossing Species

et al. 2020

View full text Add to dashboard Cite

“…Differently from grid search and random search, Bayesian optimization can save much time for determining the optimal hyper-parameters of the deep neural network [36]. The process of determining the optimal hyper-parameters of network is considered as a global optimization problem [37]. The objective function of the optimization problem can be expressed as Eq.…”

Section: B Bayesian Optimizationmentioning

confidence: 99%

Elevating Prediction Performance for Mechanical Properties of Hot-Rolled Strips by Using Semi-Supervised Regression and Deep Learning

Yang

Cao

et al. 2020

IEEE Access

View full text Add to dashboard Cite

In the present work, to solve the problem of the lacking enough labeled training data for deep learning, a safe semi-supervised regression supporting Bayesian optimization deep neural network (SAFER-BODNN) model was proposed to establish mechanical property prediction model of hot-rolled strips. The Pearson correlation coefficient was applied to reduce the data dimension. The safe semi-supervised regression was implemented to add the pseudo labels to the unlabeled data for training dataset expansion. The deep neural network was trained with Bayesian optimization to determine the optimal hyper-parameters of the network. The results show that the SAFER-BODNN model achieves good accuracy for mechanical property prediction of hot-rolled strips with R of 0.9610 for yield strength, 0.9682 for tensile strength, and 0.8619 for elongation, respectively. Compared with the deep neural network trained on the labeled dataset, the SAFER-BODNN model obtains stable smaller predicted errors. Among all the variables, C content and Mn content have large influence on the yield strength and tensile strength, coiling temperature has the largest influence on the elongation. The investigation makes full use of unlabeled data to elevate the accuracy of the deep neural network, and also provides a way for deep learning modeling when the data are insufficient.

show abstract

“…In addition to hyperparameters, the neural network architecture should be optimized for better performance. [38] In this study, because the ANN was hand-tuned by multiple trial-and-error sessions, a more effective hyperparameter set might be found by other, more sophisticated optimization methods. Finally, because the dataset was All rights reserved.…”

Section: Limitationsmentioning

confidence: 99%

Potential of machine learning to predict early ischemic events after carotid endarterectomy or stenting: A comparison with surgeon predictions

Matsuo¹,

Fujita²,

Hosoda

et al. 2020

Preprint

View full text Add to dashboard Cite

Structured AbstractObjectiveCarotid endarterectomy (CEA) and carotid artery stenting (CAS) are recommended for high stroke-risk patients with carotid artery stenosis to reduce ischemic events. However, we often face difficulty in determining the best treatment method. Therefore, it is necessary to develop a useful decision support tool to identify an appropriate patient-specific treatment for carotid artery stenosis. Our objective is to develop an accurate post-CEA/CAS outcome prediction model using machine learning (ML) algorithms that will serve as a basis for a new decision support tool for patient-specific treatment planning.MethodsRetrospectively collected data from 165 consecutive patients with carotid artery stenosis underwent CEA or CAS at a single institution were divided into training and test samples. The following six ML algorithms were tuned, and their predictive performance evaluated by comparison with surgeon predictions: an artificial neural network, logistic regression, support vector machine, Gaussian naïve Bayes, random forest, and extreme gradient boosting (XGBoost). A total of 17 clinical parameters were used for the ML model development. These parameters consisted of age, pretreatment modified Rankin scale, hypertension, diabetes mellitus, medical history of arteriosclerotic disease, serum low-density lipoprotein cholesterol value, internal carotid artery peak systolic velocity, symptomatic, crescendo transient ischemic attack or stroke in evolution, previous neck irradiation, type III aorta, contralateral carotid occlusion, stenosis at a high position, mobile plaque, plaque ulceration, vulnerable plaque, and procedure (CEA or CAS). Outcome was defined as any ischemic stroke within 30 days after treatment.ResultsThe XGBoost model performed the best in the evaluation; its sensitivity, specificity, positive predictive value, and accuracy were 66.7%, 89.5%, 50.0%, and 86.4%, respectively. The average of the outcome predictions made by four surgeons had a sensitivity of 41.7%, specificity of 75.0%, positive predictive value of 20.1%, and accuracy of 70.5%. Internal carotid artery peak systolic velocity, serum low density lipoprotein cholesterol, and procedure (CEA or CAS) were the most contributing factors according to the XGBoost algorithm.ConclusionsWe were able to develop a post-CEA/CAS outcome prediction model comparable to surgeons in performance. The accurate outcome prediction model will make it possible to make a more appropriate patient-specific selection of CEA or CAS for the treatment of carotid artery stenosis.

show abstract

Hyperparameter optimization of deep neural network using univariate dynamic encoding algorithm for searches

Cited by 107 publications

References 14 publications

Exploring Deep Learning for Complex Trait Genomic Prediction in Polyploid Outcrossing Species

Exploring Deep Learning for Complex Trait Genomic Prediction in Polyploid Outcrossing Species

Elevating Prediction Performance for Mechanical Properties of Hot-Rolled Strips by Using Semi-Supervised Regression and Deep Learning

Potential of machine learning to predict early ischemic events after carotid endarterectomy or stenting: A comparison with surgeon predictions

Contact Info

Product

Resources

About