PROTES: Probabilistic Optimization with Tensor Sampling

Abstract

We develop new method PROTES for optimization of the multidimensional arrays and discretized multivariable functions, which is based on a probabilistic sampling from a probability density function given in the low-parametric tensor train format. We tested it on complex multidimensional arrays taken, among other, from real-world applications, including unconstrained binary optimization and optimal control problems, for which the possible number of elements is up to 2 100 elements. In numerical experiments, both on analytic model functions and on complex problems, our algorithm outperform existing popular discrete optimization methods (Particle Swarm Optimization, Covariance Matrix Adaptation, Differential Evolution and others). Moreover, we take the same set of hyperparameters of our algorithm for all numerical applications.

Translate your gibberish: black-box adversarial attack on machine translation systems

“…Accordingly, in the second stage, we repeat the described optimization process, composing phrases of d (2) hallucinogens. As the possible candidates, we select n (2) (n (2) ≤ m) top hallucinogens ŵ(1) , ŵ(2) , . .…”

“…. , ŵ(n (2) ) from the result of the first stage. Without loss of generality, we have chosen d (2) = 7 and n (2) = 33, i.e., the same values as in the first stage.…”

“…, ŵ(n (2) ) from the result of the first stage. Without loss of generality, we have chosen d (2) = 7 and n (2) = 33, i.e., the same values as in the first stage. In this case, we use the loss function (2) without the second term, i.e., we do not maximize the perplexity of the input text, since it is already composed of the hallucinogens.…”

“…We formulate it as a problem of maximizing the difference between the perplexity [23] of the translation and the source text, and we apply GPT-2 [20] to define the perplexity of the input and output sequences. For a search of the best combination of input symbols we use the new global optimization method PROTES 5 [2], which is based on the low-rank tensor train (TT) decomposition [19] and can efficiently perform gradient-free multivariate discrete optimization. For all three considered MT systems, we obtained a set of sevenletter inputs in Russian that are not words, which, however, lead to a translation representing a word or set of words in English.…”