Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates

Ilievski, Ilija; Akhtar, Taimoor; Feng, Jiashi; Shoemaker, C. A.

doi:10.1609/aaai.v31i1.10647

Cited by 88 publications

(29 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The HORD algorithm was used to optimize various hyperparameters, such as batch size and initial learning rate. It was applied to the DNN-BB and the DNN-PBPK model separately.…”

Section: Materials and Methodsmentioning

confidence: 99%

Prediction of Oral Bioavailability in Rats: Transferring Insights from in Vitro Correlations to (Deep) Machine Learning Models Using in Silico Model Outputs and Chemical Structure Parameters

Schneckener

Grimbs

Hey

et al. 2019

J. Chem. Inf. Model.

View full text Add to dashboard Cite

Oral administration of drug products is a strict requirement in many medical indications. Therefore, bioavailability prediction models are of high importance for prioritization of compound candidates in the drug discovery process. However, oral exposure and bioavailability are difficult to predict, as they are the result of various highly complex factors and/or processes influenced by the physicochemical properties of a compound, such as solubility, lipophilicity, or charge state, as well as by interactions with the organism, for instance, metabolism or membrane permeation. In this study, we assess whether it is possible to predict intravenous (iv) or oral drug exposure and oral bioavailability in rats. As input parameters, we use (i) six experimentally determined in vitro and physicochemical endpoints, namely, membrane permeation, free fraction, metabolic stability, solubility, pK a value, and lipophilicity; (ii) the outputs of six in silico absorption, distribution, metabolism, and excretion models trained on the same endpoints, or (iii) the chemical structure encoded as fingerprints or simplified molecular input line entry system strings. The underlying data set for the models is an unprecedented collection of almost 1900 data points with high-quality in vivo experiments performed in rats. We find that drug exposure after iv administration can be predicted similarly well using hybrid models with in vitro-or in silico-predicted endpoints as inputs, with fold change errors (FCE) of 2.28 and 2.08, respectively. The FCEs for exposure after oral administration are higher, and here, the prediction from in vitro inputs performs significantly better in comparison to in silico-based models with FCEs of 3.49 and 2.40, respectively, most probably reflecting the higher complexity of oral bioavailability. Simplifying the prediction task to a binary alert for low oral bioavailability, based only on chemical structure, we achieve accuracy and precision close to 70%.

show abstract

“…The HORD algorithm was used to optimize various hyperparameters, such as batch size and initial learning rate. It was applied to the DNN-BB and the DNN-PBPK model separately.…”

Section: Materials and Methodsmentioning

confidence: 99%

Prediction of Oral Bioavailability in Rats: Transferring Insights from in Vitro Correlations to (Deep) Machine Learning Models Using in Silico Model Outputs and Chemical Structure Parameters

Schneckener

Grimbs

Hey

et al. 2019

J. Chem. Inf. Model.

View full text Add to dashboard Cite

show abstract

“…Dataset prediction accuracy was obtained by training a XGBoost classifier on the training set and applying to the validation set. We utilized the HORD algorithm (Regis & Shoemaker, 2013; Ilievski et al, 2017; Eriksson et al, 2020) to find the best set of hyperparameters using the validation set (Table 2). The trained DNN after 1000 epochs was utilized for subsequent analyses.…”

Section: Methodsmentioning

confidence: 99%

Goal-specific brain MRI harmonization

Chen

et al. 2022

Preprint

View full text Add to dashboard Cite

There is significant interest in pooling magnetic resonance image (MRI) data from multiple datasets to enable mega-analysis. Harmonization is typically performed to reduce heterogeneity when pooling MRI data across datasets. Most MRI harmonization algorithms do not explicitly consider downstream application performance during harmonization. However, the choice of downstream application might influence what might be considered as study-specific confounds. Therefore, ignoring downstream applications during harmonization might potentially limit downstream performance. Here we propose a goal-specific harmonization framework that utilizes downstream application performance to regularize the harmonization procedure. Our framework can be integrated with a wide variety of harmonization models based on deep neural networks, such as the recently proposed conditional variational autoencoder (cVAE) harmonization model. Three datasets from three different continents with a total of 2787 participants and 10085 anatomical T1 scans were used for evaluation. We found that cVAE removed more dataset differences than the widely used ComBat model, but at the expense of removing desirable biological information as measured by downstream prediction of mini mental state examination (MMSE) scores and clinical diagnoses. On the other hand, our goal-specific cVAE (gcVAE) was able to remove as much dataset differences as cVAE, while improving downstream cross-sectional prediction of MMSE scores and clinical diagnoses.

show abstract

“…Automatic HPO facilitates fair comparisons [20]. The HPO problem has a long history dating back to the 1990 [21]. In addition, it was determined that different hyperparameter configurations were the best results for different datasets in the early stages.…”

Section: Hyperparameter Optimizationmentioning

confidence: 99%

An intelligent demand forecasting model using a hybrid of metaheuristic optimization and deep learning algorithm for predicting concrete block production

Al-Khazraji

Nasser

Khlil

2022

IJ-AI

View full text Add to dashboard Cite

Demand forecasting aims to optimize the production planning of industrial companies by ensuring that the production planning meets the future demand. Demand forecasting utilizes historical data as an input to predict future trends of the demand. In this paper, a new approach for developing an intelligent demand forecasting model using a hybrid of metaheuristic optimization and deep learning algorithm is presented. Fireﬂy algorithmbased gated recurrent units (FA-GRU) is used to tackle the production forecasting problem. The proposed model has been evaluated and compared with the standard gated recurrent unit (GRU) and standard long short-term memory model (LSTM) using historical data of 36 months of concrete block manufacturing at dler company in Iraq. The prediction accuracy of the three models is evaluated using the root mean square error (RMSE), the mean absolute percentage error (MAPE) and the statistical coefficient of determination (R2 ) indicators. The outcomes of the study show that the proposed FA-GRU gives better forecasting results compared to the standard GRU and standard LSTM.

show abstract

Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates

Cited by 88 publications

References 16 publications

Prediction of Oral Bioavailability in Rats: Transferring Insights from in Vitro Correlations to (Deep) Machine Learning Models Using in Silico Model Outputs and Chemical Structure Parameters

Prediction of Oral Bioavailability in Rats: Transferring Insights from in Vitro Correlations to (Deep) Machine Learning Models Using in Silico Model Outputs and Chemical Structure Parameters

Goal-specific brain MRI harmonization

An intelligent demand forecasting model using a hybrid of metaheuristic optimization and deep learning algorithm for predicting concrete block production

Contact Info

Product

Resources

About