Auto-Model: Utilizing Research Papers and HPO Techniques to Deal with the CASH problem

Abstract

In many fields, a mass of algorithms with completely different hyperparameters have been developed to address the same type of problems. Choosing the algorithm and hyperparameter setting correctly can promote the overall performance greatly, but users often fail to do so due to the absence of knowledge. How to help users to effectively and quickly select the suitable algorithm and hyperparameter settings for the given task instance is an important research topic nowadays, which is known as the CASH problem. In this paper, we design the Auto-Model approach, which makes full use of known information in the related research paper and introduces hyperparameter optimization techniques, to solve the CASH problem effectively. Auto-Model tremendously reduces the cost of algorithm implementations and hyperparameter configuration space, and thus capable of dealing with the CASH problem efficiently and easily. To demonstrate the benefit of Auto-Model, we compare it with classical Auto-Weka approach. The experimental results show that our proposed approach can provide superior results and achieves better performance in a short time.

Regularized boosting with an increasing coefficient magnitude stop criterion as meta-learner in hyperparameter optimization stacking ensemble

“…Even though this task includes pre-processing, feature selection and extraction before the predictive model is induced it also calls for interpretability and decision making after the predictive model is induced. Combined Algorithms Selection and Hyperparameter Optimization (CASH) [64] is currently a top field of research that goes a step beyond the task of HPO as well, since it also selects a suitable system that provides the model in addition to the hyperparameters. However, CASH environments do not automate as many tasks as an AutoML framework.…”