A general dimension for query learning

Abstract

We introduce a combinatorial dimension that characterizes the number of queries needed to exactly (or approximately) learn concept classes in various models. Our general dimension provides tight upper and lower bounds on the query complexity for all sorts of queries, not only for example-based queries as in previous works.As an application we show that for learning DNF formulas, unspecified attribute value membership and equivalence queries are not more powerful than standard membership and equivalence queries. Further, in the approximate learning setting, we use the general dimension to characterize the query complexity in the statistical query as well as the learning by distances model. Moreover, we derive close bounds on the number of statistical queries needed to approximately learn DNF formulas.

A complete characterization of statistical query learning with applications to evolvability

“…The orthogonality-based characterization does not preserve efficiency but is more easy to analyze when proving lower bounds. Neither of these properties are possessed by the previous characterizations of strong SQ learning [3,37,38].…”

“…We describe a new and simple characterization of the query complexity of learning in the SQ learning model. Unlike the previously known bounds on SQ learning [9,11,42,3,37] our characterization preserves the accuracy and the efficiency of learning. The preservation of accuracy implies that our characterization gives the first characterization of SQ learning in the agnostic learning framework of Haussler [23], and Kearns, Schapire and Sellie [31].…”

“…A notable exception is the algorithm for learning parities of Blum et al[10] which is tolerant to random noise, albeit not in the same strong sense as the algorithms derived from SQs 2. An earlier work has also considered this question but the characterization that was obtained is in terms of queryanswering protocols that are essentially specifications of non-adaptive algorithms[3].…”

Learning definite Horn formulas from closure queries

“…Several extensions, of different focus and generality, appeared subsequently. One very general notion of query learning is that of [3]; in a less general level, some of these different extensions are of interest for this paper.…”