Robust randomized optimization with k nearest neighbors

Abstract

Modern applications of machine learning typically require the tuning of a multitude of hyperparameters. With this motivation in mind, we consider the problem of optimization given a set of noisy function evaluations. We focus on robust optimization in which the goal is to find a point in the input space such that the function remains high when perturbed by an adversary within a given radius. Here we identify the minimax optimal rate for this problem, which turns out to be of order [Formula: see text], where [Formula: see text] is the sample size and [Formula: see text] quantifies the smoothness of the function for a broad class of problems, including situations where the metric space is unbounded. The optimal rate is achieved (up to logarithmic factors) by a conceptually simple algorithm based on [Formula: see text]-nearest neighbor regression.

Automated Health Condition Diagnosis of in situ Wood Utility Poles Using an Intelligent Non-Destructive Evaluation (NDE) Framework

“…The k-NN algorithm was developed to classify the unknown data based on existing data with known labels. 32 The essence of k-NN is analogy learning, that is learning by comparing the given test sample with training samples. Suppose that in the last section, l PCs are selected from feature vector of IMF energy ratios as the inputs of the ML model, which indicates that all the training samples should be in this l-dimensional space.…”