High-Dimensional Dueling Optimization with Preference Embedding

Abstract

In many scenarios of black-box optimization, evaluating the objective function values of solutions is expensive, while comparing a pair of solutions is relatively cheap, which yields the dueling black-box optimization. The side effect of dueling optimization is that it doubles the dimension of solution space and exacerbates the dimensionality scalability issue of black-box optimization, e.g., Bayesian optimization. To address this issue, the existing dueling optimization methods fix one solution when dueling throughout the optimization process, but it may reduce their efficacy. Fortunately, it has been observed that, in recommendation systems, the dueling results are mainly determined by the latent human preferences. In this paper, we abstract this phenomenon as the preferential intrinsic dimension and inject it into the dueling Bayesian optimization, resulting in the preferential embedding dueling Bayesian optimization (PE-DBO). PE-DBO decouples optimization and pairwise comparison via the preferential embedding matrix. Optimization is performed in the preferential intrinsic subspace with much lower dimensionality, while pairwise comparison is completed in the original dueling solution space. Theoretically, we disclose that the preference function can be approximately preserved in the lower-dimensional preferential intrinsic subspace. Experiment results verify that, on molecule discovery and web page recommendation dueling optimization tasks, the preferential intrinsic dimension exists and PE-DBO is superior in scalability compared with that of the state-of-the-art (SOTA) methods.

Degradation-Resistant Offline Optimization via Accumulative Risk Control

“…Bayesian optimization (BO) [31,25] under the offline scenario explicitly assesses uncertainty by the Gaussian process surrogate model and leverages it in the acquisition function to alleviate the performance degradation in optimization. Although the latest BO studies have made significant progress in addressing the curse of dimensionality under certain conditions [26,36,4], they still have difficulties in addressing large-scale offline data. Some existing methods propose to sample solutions from learned generative models and introduce regularization in the sampling process.…”