Practical Design Space Exploration

Abstract

Multi-objective optimization is a crucial matter in computer systems design space exploration because real-world applications often rely on a trade-off between several objectives. Derivatives are usually not available or impractical to compute and the feasibility of an experiment can not always be determined in advance. These problems are particularly difficult when the feasible region is relatively small, and it may be prohibitive to even find a feasible experiment, let alone an optimal one.We introduce a new methodology and corresponding software framework, HyperMapper 2.0, which handles multi-objective optimization, unknown feasibility constraints, and categorical/ordinal variables. This new methodology also supports injection of the user prior knowledge in the search when available. All of these features are common requirements in computer systems but rarely exposed in existing design space exploration systems. The proposed methodology follows a white-box model which is simple to understand and interpret (unlike, for example, neural networks) and can be used by the user to better understand the results of the automatic search.We apply and evaluate the new methodology to the automatic static tuning of hardware accelerators within the recently introduced Spatial programming language, with minimization of design run-time and compute logic under the constraint of the design fitting in a target field-programmable gate array chip. Our results show that HyperMapper 2.0 provides better Pareto fronts compared to state-of-the-art baselines, with better or competitive hypervolume indicator and with 8x improvement in sampling budget for most of the benchmarks explored.

Data-driven software design with Constraint Oriented Multi-variate Bandit Optimization (COMBO)

“…Hundreds of metrics are reportedly used for a single experiment at Microsoft (Kevic et al 2017;Machmouchi and Buscher 2016). For optimization, multi-objective optimization (Nardi et al 2019;Sun et al 2018) can be applied offline at compile time, where someone can manually make a trade-off between metrics from a Pareto front. However, for the online bandit optimization algorithms, a single metric is required to serve as rewards (though that metric can be a scalar index).…”

Processamento de linguagem natural aplicado à classificação de decretos administrativos brasileiros

“…A seguir, treinamos o algoritmo SVM de kernel linear aplicando o método de relevância binária para obter classificações multi-class e multi-label. Nesse processo, otimizamos ainda o hiperparâmetro de regularização do SVM (C) através de 35 iterações de otimização bayesiana [6].…”

“…Nessas abordagens, as palavras são representadas através de embeddings. Otimizamos ainda os principais hiperparâmetros dessas redes através de 100 iterações de otimização bayesiana [6]. Após obtidos, utilizamos os melhores hiperparâmetros para treinar o modelo final e testá-lo.…”

Polystore++: Accelerated Polystore System for Heterogeneous Workloads

“…HyperMapper 2.0 [60], [61], [62] is designed for heterogeneous workloads, and can handle complex design spaces consisting of multiple objectives, categorical/ordinal variables, unknown feasibility constraints [63], and exploitation of performance profiling of earlier executions of workloads in Polystore++.…”

“…Moreover, building cost models with a large number of parameters may be expensive. Alternatively, active learning [61], [62], [60] can trade off exploration and exploitation mechanisms to give an approximated optimal configuration for workload execution in Polystore++ systems.…”