Incorporating domain knowledge into neural-guided search

Abstract

Many AutoML problems involve optimizing discrete objects under a black-box reward. Neural-guided search provides a flexible means of searching these combinatorial spaces using an autoregressive recurrent neural network. A major benefit of this approach is that builds up objects sequentially-this provides an opportunity to incorporate domain knowledge into the search by directly modifying the logits emitted during sampling. In this work, we formalize a framework for incorporating such in situ priors and constraints into neuralguided search, and provide sufficient conditions for enforcing constraints. We integrate several priors and constraints from existing works into this framework, propose several new ones, and demonstrate their efficacy in informing the task of symbolic regression.

Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws

“…However, in this work we show that respecting physical constraints actually helps improve SR performance not only in terms of interpretability but also in accuracy by guiding the exploration of the space of solutions towards exact analytical laws. This is consistent with the studies of Petersen et al (2019Petersen et al ( , 2021; Kammerer et al (2020) who found that using in situ constraints during analytical expression generation is much more efficient as it vastly reduces the search space of trial expressions (though we note their machinery is not capable of incorporating units constraints as it requires one to compute and exploit the whole graph describing an expression as a relational tree).…”

“…units prior was hinted by Petersen et al (2021), to the best of our knowledge such a framework was never built before.…”

“…It is important to note here that one can artificially tune the generated categorical distribution to incorporate prior knowledge in situ while expressions are being generated. One can for example zero-out the probability of some token depending on the context encoded in the expression tree being generated, thus greatly reducing search space (Petersen et al 2019(Petersen et al , 2021. We therefore adopt priors that force expressions sizes to be < 30 tokens long, to contain no more than 2 levels of nested trigonometric operations (e.g., forbidding cos(f.t + sin(x/x0 + tan( ))) but still allowing cos(f.t+sin(x/x0))), contain no self nesting of exponent and log operators (e.g., forbidding e e ) and forbid useless inverse unary operations (e.g., forbidding e log ).…”