Faster Model-Based Optimization Through Resource-Aware Scheduling Strategies

Richter, Jakob; Kotthaus, Helena; Bischl, Bernd; Marwedel, Peter; Rahnenführer, Jörg; Lang, Michel

doi:10.1007/978-3-319-50349-3_22

Cited by 9 publications

(11 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, Coy et al (2020) proposed the parallelized Bayesian optimization by keeping the number of evaluations low (sample efficient) and executing parallel evaluations to reduce wall-clock time, which outperforms the state-of-the-art parallel CMA-ES (Hansen and Ostermeier 2001) even on higher dimensions, e.g., 20-dimensional Sharp Ridge function. To account for heterogeneous run-times of different proposals, asynchronous parallel strategies (Janusevskis et al 2012) as well as scheduling methods (Richter et al 2016;Kotthaus et al 2019) have been developed.…”

Section: Hyper-parameter Tuning Algorithmsmentioning

confidence: 99%

“…Hence the improvement of efficiency is actually less than n times. Although asynchronous parallel strategies (Janusevskis et al 2012) as well as scheduling methods (Richter et al 2016;Kotthaus et al 2019) are developed for heterogeneous run-time of different proposals, the comparison of different surrogate updating strategies is considered out of scope. When there are many nodes, the resulting surrogate may not be able to generate a sufficient number of valuable proposals for evaluating the machine learning algorithms in parallel in the next iteration.…”

Section: Comparison Between Modes -B and Modes-imentioning

confidence: 99%

See 1 more Smart Citation

MODES: model-based optimization on distributed embedded systems

et al. 2021

Self Cite

View full text Add to dashboard Cite

The predictive performance of a machine learning model highly depends on the corresponding hyper-parameter setting. Hence, hyper-parameter tuning is often indispensable. Normally such tuning requires the dedicated machine learning model to be trained and evaluated on centralized data to obtain a performance estimate. However, in a distributed machine learning scenario, it is not always possible to collect all the data from all nodes due to privacy concerns or storage limitations. Moreover, if data has to be transferred through low bandwidth connections it reduces the time available for tuning. Model-Based Optimization (MBO) is one state-of-the-art method for tuning hyper-parameters but the application on distributed machine learning models or federated learning lacks research. This work proposes a framework $$\textit{MODES}$$ MODES that allows to deploy MBO on resource-constrained distributed embedded systems. Each node trains an individual model based on its local data. The goal is to optimize the combined prediction accuracy. The presented framework offers two optimization modes: (1) $$\textit{MODES}$$ MODES -B considers the whole ensemble as a single black box and optimizes the hyper-parameters of each individual model jointly, and (2) $$\textit{MODES}$$ MODES -I considers all models as clones of the same black box which allows it to efficiently parallelize the optimization in a distributed setting. We evaluate $$\textit{MODES}$$ MODES by conducting experiments on the optimization for the hyper-parameters of a random forest and a multi-layer perceptron. The experimental results demonstrate that, with an improvement in terms of mean accuracy ($$\textit{MODES}$$ MODES -B), run-time efficiency ($$\textit{MODES}$$ MODES -I), and statistical stability for both modes, $$\textit{MODES}$$ MODES outperforms the baseline, i.e., carry out tuning with MBO on each node individually with its local sub-data set.

show abstract

Section: Hyper-parameter Tuning Algorithmsmentioning

confidence: 99%

Section: Comparison Between Modes -B and Modes-imentioning

confidence: 99%

MODES: model-based optimization on distributed embedded systems

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…In a formal way, a learning algorithm processes a training set

τ = false(τ_{i}, y_{i} false);

τ_{i} \in τ

and

y_{i} \in false[1, - 1 false]; i = 1 \dots n

comprising n examples

false(τ_{i}, y_{i} false)

, where

τ_{i}

represents a task input and

y_{i}

is the associated output. The output of this prediction can be a numerical value for the regression model or a class label for the classification model [20].…”

Section: Analysis Of Scheduling Algorithms and Proposed Framework Fmentioning

confidence: 99%

“…where τ i represents a task input and y i is the associated output. The output of this prediction can be a numerical value for the regression model or a class label for the classification model [20]. The proposed approach aims to use Naive Bayes classifier [21] to estimate the syntactical and semantic similarity of the execution pattern of low-and high-criticality tasks.…”

Section: Classifiermentioning

confidence: 99%

Extending resources for avoiding overloads of mixed‐criticality tasks in cyber‐physical systems

Maruf¹,

Azim²

2019

IET cyber-phys. syst.

View full text Add to dashboard Cite

“…Furthermore, expensive evaluation times may vary, e.g., CFD simulations may differ in time consumption, depending on the evaluated candidate solution. To that end, Richter et al [14] propose an asynchronous approach that attempts to produce evaluation schedules that reduce the overall time consumption. They use surrogate models to approximate the objective function results as well as to approximate the required resources.…”

Section: Further Approachesmentioning

confidence: 99%

Comparison of parallel surrogate-assisted optimization approaches

Rehbach

Zaefferer

Stork

et al. 2018

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

The availability of several CPU cores on current computers enables parallelization and increases the computational power significantly. Optimization algorithms have to be adapted to exploit these highly parallelized systems and evaluate multiple candidate solutions in each iteration. This issue is especially challenging for expensive optimization problems, where surrogate models are employed to reduce the load of objective function evaluations. This paper compares different approaches for surrogate modelbased optimization in parallel environments. Additionally, an easy to use method, which was developed for an industrial project, is proposed. All described algorithms are tested with a variety of standard benchmark functions. Furthermore, they are applied to a real-world engineering problem, the electrostatic precipitator problem. Expensive computational fluid dynamics simulations are required to estimate the performance of the precipitator. The task is to optimize a gas-distribution system so that a desired velocity distribution is achieved for the gas flow throughout the precipitator. The vast amount of possible configurations leads to a complex discrete valued optimization problem. The experiments indicate that a hybrid approach works best, which proposes candidate solutions based on different surrogate model-based infill criteria and evolutionary operators. CCS CONCEPTS • Mathematics of computing → Discrete optimization; • Theory of computation → Continuous optimization; Gaussian processes; • Computing methodologies → Modeling and simulation;

show abstract

Faster Model-Based Optimization Through Resource-Aware Scheduling Strategies

Cited by 9 publications

References 8 publications

MODES: model-based optimization on distributed embedded systems

MODES: model-based optimization on distributed embedded systems

Extending resources for avoiding overloads of mixed‐criticality tasks in cyber‐physical systems

Comparison of parallel surrogate-assisted optimization approaches

Contact Info

Product

Resources

About