A Framework for Selecting Deep Learning Hyper-parameters

Donoghue, Jim O'; Roantree, Mark

doi:10.1007/978-3-319-20424-6_12

Cited by 6 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ANN algorithm is also known to produce good accuracy in energy load prediction (K. Li et al, 2018). Multi-layer Perceptron (MLP) is a function of a deep neural network that utilizes a feed forward propagation process with one hidden layer where latent and abstract features are learned (Donoghue and Roantree, 2015). In research by Khantach et al, Multi-layer Perceptron ANN produced the most accurate result between support vector machine (SVM), Gaussian process and radial basis function (RBF) with a Mean Absolute Percentage Error (MAPE) of 0.96 (Khantach et al, 2019).…”

Section: Literature Reviewmentioning

confidence: 99%

Building energy consumption prediction for residential buildings using deep learning and other machine learning techniques

Olu-Ajayi

Alaka

Sulaimon

et al. 2022

Journal of Building Engineering

211

View full text Add to dashboard Cite

Section: Literature Reviewmentioning

confidence: 99%

Building energy consumption prediction for residential buildings using deep learning and other machine learning techniques

Olu-Ajayi

Alaka

Sulaimon

et al. 2022

Journal of Building Engineering

211

View full text Add to dashboard Cite

“…The discarded works did not comply with the quality criteria for a series of different reasons. Some papers, for example, were more focused on hyperparameter optimization (out of the scope of this review) than in selecting an appropriate algorithm given the context of application [53][54][55][56][57][58].…”

Section: Clarifications On the Excluded Recordsmentioning

confidence: 99%

Explainable Rules and Heuristics in AI Algorithm Recommendation Approaches—A Systematic Literature Review and Mapping Study

Jos�Garc韆-Pe馻lvo¹,

V醶quez-Ingelmo²,

Garc韆-Holgado³

2023

Computer Modeling in Engineering &Amp; Sciences

View full text Add to dashboard Cite

The exponential use of artificial intelligence (AI) to solve and automated complex tasks has catapulted its popularity generating some challenges that need to be addressed. While AI is a powerful means to discover interesting patterns and obtain predictive models, the use of these algorithms comes with a great responsibility, as an incomplete or unbalanced set of training data or an unproper interpretation of the models' outcomes could result in misleading conclusions that ultimately could become very dangerous. For these reasons, it is important to rely on expert knowledge when applying these methods. However, not every user can count on this specific expertise; non-AIexpert users could also benefit from applying these powerful algorithms to their domain problems, but they need basic guidelines to obtain the most out of AI models. The goal of this work is to present a systematic review of the literature to analyze studies whose outcomes are explainable rules and heuristics to select suitable AI algorithms given a set of input features. The systematic review follows the methodology proposed by Kitchenham and other authors in the field of software engineering. As a result, 9 papers that tackle AI algorithm recommendation through tangible and traceable rules and heuristics were collected. The reduced number of retrieved papers suggests a lack of reporting explicit rules and heuristics when testing the suitability and performance of AI algorithms.

show abstract

“…Unfortunately, despite the fact that deep learning algorithms have been around for a long time, there are no wellestablished procedures for hyper-parameters tuning, such as back-propagation for a model training [5]. Instead, a set of custom techniques, such as grid, random and heuristic search [6,7], have been developed and used by most of DL systems designers.…”

Section: Introductionmentioning

confidence: 99%

The observer-assisted method for adjusting hyper-parameters in deep learning algorithms

Wielgosz

2016

Preprint

View full text Add to dashboard Cite

This paper presents a concept of a novel method for adjusting hyper-parameters in Deep Learning (DL) algorithms. An external agent-observer monitors a performance of a selected Deep Learning algorithm. The observer learns to model the DL algorithm using a series of random experiments. Consequently, it may be used for predicting a response of the DL algorithm in terms of a selected quality measurement to a set of hyper-parameters. This allows to construct an ensemble composed of a series of evaluators which constitute an observer-assisted architecture. The architecture may be used to gradually iterate towards to the best achievable quality score in tiny steps governed by a unit of progress. The algorithm is stopped when the maximum number of steps is reached or no further progress is made.

show abstract

A Framework for Selecting Deep Learning Hyper-parameters

Cited by 6 publications

References 17 publications

Building energy consumption prediction for residential buildings using deep learning and other machine learning techniques

Building energy consumption prediction for residential buildings using deep learning and other machine learning techniques

Explainable Rules and Heuristics in AI Algorithm Recommendation Approaches—A Systematic Literature Review and Mapping Study

The observer-assisted method for adjusting hyper-parameters in deep learning algorithms

Contact Info

Product

Resources

About