Predicting execution time of machine learning tasks for scheduling

Toward a Progress Indicator for Machine Learning Model Building and Data Mining Algorithm Execution

“…Researchers have done much work on predicting machine learning model building time [15, 59–61, 63, 68] and the runtime of a computer program [20, 25, 29, 65, 74], an iterative algorithm on graph data [57], and an algorithm for solving a combinatorial optimization problem [30]. The predicted model building time is often inaccurate and may differ considerably from the actual model building time on a loaded computer.…”

“…The predictive model forecasts the number of needed iterations based on a data set’s feature values and the algorithm’s hyper-parameter values. Meta-learning was used before to predict model building time [15, 59–61, 63, 68]. …”

“…For tasks such as scheduling machine learning model building jobs [59] and automatic machine learning model selection [10, 17, 18, 40, 68], meta-learning has been used before to predict model accuracy and building time. There, no constraint is put on the hyper-parameter value combinations selected for testing.…”

“…So far, sophisticated, determinate progress indicators have been designed for automatic machine learning model selection [41, 46] and built for database queries [11, 37, 43–45], program compilation [42], static program analysis [38], subgraph queries [77], and MapReduce jobs [47, 48]. Besides making the software more user friendly, determinate progress indicators can also help with workload management [42, 45, 59]. In the rest of the paper, we focus on determinate progress indicators, unless indicated otherwise.…”

“…Another way to offer a trivial progress indicator is to use an existing method to predict a machine learning model’s building time before model building starts [15, 59–61, 63, 68]. If the model building task is predicted to take t seconds and has run for t ′ seconds, the remaining time is projected to be t-t ′ seconds.…”