Hyper-parameter Optimization for Latent Spaces

Abstract

We present an online optimization method for time-evolving data streams that can automatically adapt the hyper-parameters of an embedding model. More specifically, we employ the Nelder-Mead algorithm, which uses a set of heuristics to produce and exploit several potentially good configurations, from which the best one is selected and deployed. This step is repeated whenever the distribution of the data is changing. We evaluate our approach on streams of real-world as well as synthetic data, where the latter is generated in such way that its characteristics change over time (concept drift). Overall, we achieve good performance in terms of accuracy compared to state-of-the-art AutoML techniques.

Auto-tuning HyperParameters of SGD Matrix Factorization-Based Recommender Systems Using Genetic Algorithm

“…The results presented in [7], [8] corroborate the importance of hyperparameter optimization in recommender systems. Some works in the literature suggest algorithms to address this problem, such as 1)Random search [9], 2)Gradient Search [10], 3)Bayesian Optimization, and Online Optimization [11].…”