aMLLibrary: An AutoML Approach For Performance Prediction

Abstract

aMLLibrary is an open-source, high-level Python package that allows the parallel building of multiple Machine Learning (ML) regression models. It is focused on performance modeling and supports several methods for feature engineering/selection and hyperparameter tuning. The library implements fault tolerance mechanisms to recover from system crashes, and only a simple declarative text file is required to launch a full experimental campaign for all required models. Its modular structure allows users to implement their own plugins and model-building wrappers and easily add them to the library. We test aMLLibrary on building the performance models of neural networks and image processing applications, with the best model produced often having less than 20% prediction error.

Performance Models for Distributed Deep Learning Training Jobs on Ray

“…The reported numbers of CPUs and GPUs in Table I are intended per-worker. For each experiment, we collected the execution times of different stages, as shown in Figure 2: 2) Performance Models Building: Machine Learning (ML)based performance models were built through the aMLLibrary, an open-source tool for the automatic generation of regression models proposed in [29]. Specifically, the set of considered ML methods and corresponding hyperparameters is reported in Table II.…”