Toward a Progress Indicator for Machine Learning Model Building and Data Mining Algorithm Execution

Abstract

For user-friendliness, many software systems offer progress indicators for long-duration tasks. A typical progress indicator continuously estimates the remaining task execution time as well as the portion of the task that has been finished. Building a machine learning model often takes a long time, but no existing machine learning software supplies a non-trivial progress indicator. Similarly, running a data mining algorithm often takes a long time, but no existing data mining software provides a nontrivial progress indicator. In this article, we consider the problem of offering progress indicators for machine learning model building and data mining algorithm execution. We discuss the goals and challenges intrinsic to this problem. Then we describe an initial framework for implementing such progress indicators and two advanced, potential uses of them, with the goal of inspiring future research on this topic.

Progress Indication for Deep Learning Model Training: A Feasibility Demonstration

“…In this section, we briefly review the related work. A detailed discussion of the related work is available in our prior paper [7].…”

“…For machine learning model training, we have built sophisticated progress indicators for decision tree, random forest, and neural network when the number of epochs needed for model training is known beforehand [7], [11]. In addition, sophisticated progress indicators have been proposed for database queries [8], [14]- [17], static program analysis [18], program compilation [19], subgraph queries [20], MapReduce jobs [21], [22], and automatic machine learning model selection [23], [24].…”

“…In addition, sophisticated progress indicators have been proposed for database queries [8], [14]- [17], static program analysis [18], program compilation [19], subgraph queries [20], MapReduce jobs [21], [22], and automatic machine learning model selection [23], [24]. As each kind of task has its own unique properties, we cannot directly adopt the existing techniques [7], [8], [11], [14]- [24] to implement progress indicators for deep learning model training when early stopping is allowed.…”

“…Besides making the deep learning model training process more user-friendly, we can use the information given by the progress indicator to assist with workload management as outlined in our papers [7], [8]. We recently talked with Yasser M. Ibrahim, the head of distributed machine learning at Amazon.…”

“…As the amount of training data, the neural network's hyper-parameter values, and the server capacity continue changing over time, his team needs a method to find an appropriate cluster configuration for each round of re-training. A workload management approach aided by progress indicators would serve this purpose [7].…”