Correlated Multi-objective Multi-fidelity Optimization for HLS Directives Design

Sun, Qi; Chen, Tinghuan; Liu, Siting; Jiang, Miao; Chen, Jianli; Yu, Hao; Yu, Bei

doi:10.23919/date51398.2021.9474241

Cited by 23 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, although BO has been extended to deal with multiobjectives (Svenson & Santner, 2016;Feliot et al, 2017;Yang et al, 2019;Iqbal et al, 2020;Daulton et al, 2020) as well as multiple fidelities and multiple information sources (Lam et al, 2015;Poloczek et al, 2017;Ghoreishi & Allaire, 2019;Candelieri & Archetti, 2021b;a;Ariafar et al, 2021), there is a significant lack of solutions jointly addressing the two tasks. On the other hand, the research interest on this specific challenge is quickly increasing, especially because its applicability to many other real-life problems than fair and green ML, as demonstrated by very recent works such as (Sun et al, 2022) and (Irshad et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

Fair and Green Hyperparameter Optimization via Multi-objective and Multiple Information Source Bayesian Optimization

Candelieri¹,

Ponti²,

Archetti³

2022

Preprint

View full text Add to dashboard Cite

There is a consensus that focusing only on accuracy in searching for optimal machine learning models amplifies biases contained in the data, leading to unfair predictions and decision supports. Recently, multi-objective hyperparameter optimization has been proposed to search for machine learning models which offer equally Paretoefficient trade-offs between accuracy and fairness. Although these approaches proved to be more versatile than fairness-aware machine learning algorithms -which optimize accuracy constrained to some threshold on fairness -they could drastically increase the energy consumption in the case of large datasets. In this paper we propose FanG-HPO, a Fair and Green Hyperparameter Optimization (HPO) approach based on both multi-objective and multiple information source Bayesian optimization. FanG-HPO uses subsets of the large dataset (aka information sources) to obtain cheap approximations of both accuracy and fairness, and multi-objective Bayesian Optimization to efficiently identify Pareto-efficient machine learning models. Experiments consider two benchmark (fairness) datasets and two machine learning algorithms (XGBoost and Multi-Layer Perceptron), and provide an assessment of FanG-HPO against both fairness-aware machine learning algorithms and hyperparameter optimization via a multi-objective single-source optimization algorithm in BoTorch, a state-of-the-art platform for Bayesian Optimization.

show abstract

Section: Related Workmentioning

confidence: 99%

Fair and Green Hyperparameter Optimization via Multi-objective and Multiple Information Source Bayesian Optimization

Candelieri¹,

Ponti²,

Archetti³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…As stated in Section 1, the overall computational efficiency of BO can be increased by leveraging inexpensive LF datasets. MF BO has been successfully used in many applications such as hyperparameter tuning [41][42][43][44], finding Pareto fronts in multi-objective optimizations [45][46][47], and solving non-linear state-space models [48,49]. For MFBO, both the emulator and the AF must accommodate the multi-source and unbalanced 5 nature of the data.…”

Section: Existing Multi-fidelity Bo Techniquesmentioning

confidence: 99%

Multi-Fidelity Cost-Aware Bayesian Optimization

Foumani¹,

Shishehbor²,

Yousefpour³

et al. 2022

Preprint

View full text Add to dashboard Cite

Bayesian optimization (BO) is increasingly employed in critical applications such as materials design and drug discovery. An increasingly popular strategy in BO is to forgo the sole reliance on high-fidelity data and instead use an ensemble of information sources which provide inexpensive low-fidelity data. The overall premise of this strategy is to reduce the overall sampling costs by querying inexpensive low-fidelity sources whose data are correlated with high-fidelity samples. Here, we propose a multi-fidelity cost-aware BO framework that dramatically outperforms the state-of-the-art technologies in terms of efficiency, consistency, and robustness. We demonstrate the advantages of our framework on analytic and engineering problems and argue that these benefits stem from our two main contributions: (1) we develop a novel acquisition function for multi-fidelity cost-aware BO that safeguards the convergence against the biases of low-fidelity data, and(2) we tailor a newly developed emulator for multi-fidelity BO which enables us to not only simultaneously learn from an ensemble of multi-fidelity datasets, but also identify the severely biased low-fidelity sources that should be excluded from BO.

show abstract