Training a 3-node neural network is NP-complete

Blum, Avrim; Rivest, Ronald L.

doi:10.1016/s0893-6080(05)80010-3

Cited by 554 publications

(368 citation statements)

References 4 publications

Supporting

Mentioning

359

Contrasting

Unclassified

Order By: Relevance

“…This was used to show that if the consistency problem for F is NP-hard, then F is not PAC-learnable unless RP = NP. The technique has seen a number of applications [51], [15], [26], [14]. Because learning with equivalence queries implies PAC learning, it follows that for such classes F , F cannot be learned with equivalence queries unless RP = NP.…”

Section: Complexity Theory and Learningmentioning

confidence: 99%

“…When k ≥ 2, e.g., for learning the union of two halfspaces (or neural nets with two hidden nodes and one root-level "OR"), Blum and Rivest showed that the consistency problem is NP-hard [14], and thus this class of functions cannot be learned in polynomial time in the PAC model if RP = NP, nor in the equivalence query model if P = NP. Whether or not the union of two halfspaces can be "properly" learned (i.e., using hypotheses that are unions of two halfspaces) in polynomial time when membership queries are also allowed remains an open question in both the PAC and the equivalence query model.…”

Section: Applications To Learningmentioning

confidence: 99%

“…The intersection of k halfspaces, for constant k ≥ 2, and dually, the union of k halfspaces (simple generalization of [14]). …”

Section: Some Classes Not Learnable In Polytime With Equivalence Queriesmentioning

confidence: 99%

See 2 more Smart Citations

Complexity theoretic hardness results for query learning

et al. 1998

View full text Add to dashboard Cite

We investigate the complexity of learning for the well-studied model in which the learning algorithm may ask membership and equivalence queries. While complexity theoretic techniques have previously been used to prove hardness results in various learning models, these techniques typically are not strong enough to use when a learning algorithm may make membership queries. We develop a general technique for proving hardness results for learning with membership and equivalence queries (and for more general query models). We apply the technique to show that, assuming NP = co-NP, no polynomial-time membership and (proper) equivalence query algorithms exist for exactly learning read-thrice DNF formulas, unions of k ≥ 3 halfspaces over the Boolean domain, or some other related classes. Our hardness results are representation dependent, and do not preclude the existence of representation independent algorithms.The general technique introduces the representation problem for a class F of representations (e.g., formulas), which is naturally associated with the learning problem for F . This problem is related to the structural question of how to characterize functions representable by formulas in F , and is a generalization of standard complexity problems such as Satisfiability. While in general the representation problem is in Σ P 2 , we present a theorem demonstrating that for "reasonable" classes F , the existence of a polynomial-time membership and equivalence query algorithm for exactly learning F implies that the representation problem for F is in fact in co-NP. The theorem is applied to prove hardness results such as the ones mentioned above, by showing that the representation problem for specific classes of formulas is NP-hard.

show abstract

Section: Complexity Theory and Learningmentioning

confidence: 99%

Section: Applications To Learningmentioning

confidence: 99%

See 1 more Smart Citation

Complexity theoretic hardness results for query learning

et al. 1998

View full text Add to dashboard Cite

show abstract

“…Blum and Rivest [15] show that finding an intersection of two halfspaces that are consistent with a sample of labeled points from the boolean domain, if it exists, is NP-complete. Baum [16] presents an algorithm that learns intersections of two halfspaces from examples and membership queries, or from examples alone if the distribution obeys a symmetry condition.…”

Section: Background and Relatedmentioning

confidence: 99%

“…POLLY will actually produce no more hyperplanes than the number in the target. This is the task found NPhard when the domain is restricted to the boolean hypercube in the PAC model without membership queries [15], and in the more demanding exact learning model with both membership and equivalence queries [19], [20].…”

Section: The Forestmentioning

confidence: 99%

PAC Learning Intersections of Halfspaces with Membership Queries

Kwek¹,

Pitt²

1998

Algorithmica

View full text Add to dashboard Cite

A randomized learning algorithm POLLY is presented that efficiently learns intersections of s halfspaces in n dimensions, in time polynomial in both s and n. The learning protocol is the PAC (probably approximately correct) model of Valiant, augmented with membership queries. In particular, POLLY receives a set S of m = poly(n, s, 1/ε, 1/δ) randomly generated points from an arbitrary distribution over the unit hypercube, and is told exactly which points are contained in, and which points are not contained in, the convex polyhedron P defined by the halfspaces. POLLY may also obtain the same information about points of its own choosing. It is shown that after poly(n, s, 1/ε, 1/δ, log(1/d)) time, the probability that POLLY fails to output a collection of s halfspaces with classification error at most ε, is at most δ. Here, d is the minimum distance between the boundary of the target and those examples in S that are not lying on the boundary. The parameter log(1/d) can be bounded by the number of bits needed to encode the coefficients of the bounding hyperplanes and the coordinates of the sampled examples S. Moreover, POLLY can be extended to learn unions of k disjoint polyhedra with each polyhedron having at most s facets, in time poly(n, k, s, 1/ε, 1/δ, log(1/d), 1/γ ) where γ is the minimum distance between any two distinct polyhedra.

show abstract