Theory and Practice of Super Parameter Optimization for Machine Learning Algorithm

Abstract

Before the rise of large machine learning algorithms, most people manually adjusted the super parameters of the model by relying on experience. However, with the increasing complexity of the model, this method obviously cannot meet the needs. This paper mainly studies the theory and practice of super parameter optimization of machine learning algorithm. This thesis proposes a regression-based hyperparameter optimization algorithm that has the same data-based optimization algorithm as the optimization algorithm Bayesian. The optimization algorithm is based on the Gaussian regression process. In addition to being affected by the super parameters of the kernel function in the process of GP regression fitting, the calculation amount of the algorithm will also increase significantly. The experimental results show that, compared to the optimization algorithm, the parameter optimization results of this algorithm are similar to those of the optimization algorithm.