Learnability beyond AC ⁰

Abstract

We give an algorithm to learn constant-depth polynomialsize circuits augmented with majority gates under the uniform distribution using random examples only. For circuits which contain a polylogarithmic number of majority gates the algorithm runs in quasipolynomial time. This is the first algorithm for learning a more expressive circuit class than the class AC 0 of constant-depth polynomial-size circuits, a class which was shown to be learnable in quasipolynomial time by Linial, Mansour and Nisan in 1989. Our approach combines an extension of some of the Fourier analysis from Linial et al. with hypothesis boosting. We also show that under a standard cryptographic assumption our algorithm is essentially optimal with respect to both running time and expressiveness (number of majority gates) of the circuits being learned.

Learning a circuit by injecting values

“…Positive learnability results include those for fairly limited classes, including proposi-tional Horn formulas [2] general read once Boolean formulas [3], and decision trees [5], and those for specific distributions, including AC0 circuits [12], DNF formulas [8] and AC0 circuits with a limited number of majority gates [9]. (Note that algorithms in both papers [12] and [9] for learning AC0 circuits and their variants run only in quasipolynomial time.) Valiant gives cryptographic evidence for the difficulty of PAC learning general Boolean circuits [15].…”