We propose a theoretical understanding of neural networks in terms of Wilsonian effective field theory. The correspondence relies on the fact that many asymptotic neural networks are drawn from Gaussian processes (GPs), the analog of non-interacting field theories. Moving away from the asymptotic limit yields a non-Gaussian process (NGP) and corresponds to turning on particle interactions, allowing for the computation of correlation functions of neural network outputs with Feynman diagrams. Minimal NGP likelihoods are determined by the most relevant non-Gaussian terms, according to the flow in their coefficients induced by the Wilsonian renormalization group. This yields a direct connection between overparameterization and simplicity of neural network likelihoods. Whether the coefficients are constants or functions may be understood in terms of GP limit symmetries, as expected from ’t Hooft’s technical naturalness. General theoretical calculations are matched to neural network experiments in the simplest class of models allowing the correspondence. Our formalism is valid for any of the many architectures that becomes a GP in an asymptotic limit, a property preserved under certain types of training.

Parameter-space and function-space provide two different duality frames in which to study neural networks. We demonstrate that symmetries of network densities may be determined via dual computations of network correlation functions, even when the density is unknown and the network is not equivariant. Symmetry-viaduality relies on invariance properties of the correlation functions, which stem from the choice of network parameter distributions. Input and output symmetries of neural network densities are determined, which recover known Gaussian process results in the infinite width limit. The mechanism may also be utilized to determine symmetries during training, when parameters are correlated, as well as symmetries of the Neural Tangent Kernel. We demonstrate that the amount of symmetry in the initialization density affects the accuracy of networks trained on Fashion-MNIST, and that symmetry breaking helps only when it is in the direction of ground truth. * Equal contributions by Maiti and Stoner.Preprint. Under review.

Dark Yang-Mills sectors, which are ubiquitous in the string landscape, may be reheated above their critical temperature and subsequently go through a confining first-order phase transition that produces stochastic gravitational waves in the early universe. Taking into account constraints from lattice and from Yang-Mills (center and Weyl) symmetries, we use a phenomenological model to construct an effective potential of the semi quark-gluon plasma phase, from which we compute the gravitational wave signal produced during confinement for numerous gauge groups. The signal is maximized when the dark sector dominates the energy density of the universe at the time of the phase transition. In that case, we find that it is within reach of the next-to-next generation of experiments (BBO, DECIGO) for a range of dark confinement scales near the weak scale.

No abstract