Ignoring data may be the only way to learn efficiently

Abstract

In designing learning algorithms it seems quite reasonable to construct them in a way such that all data the algorithm already has obtained are correctly and completely reflected in the hypothesis the algorithm outputs on these data. However, this approach may totally fail, i.e. it may lead to the unsolvability of the learning problem, or it may exclude any efficient solution of it. In particular, we present a natural learning problem and prove that it can be solved in polynomial time if and only if the algorithm is allowed to ignore data.

Learning and consistency

“…As it follows from Theorem 3 and Theorem 4, for any of the learning types LT ∈ {T −CON S, R−CON S, CON S}, we have NUM ⊂ LT . On the other hand, as it was shown in Wiehagen and Zeugmann (1994), for any class U ⊆ R and any numbering ψ ∈ P 2 , if U / ∈ NUM and U ⊆ P ψ , then the halting problem with respect to ψ is undecidable, i.e., there is no h ∈ R 2 such that for any i, x ∈ IN, h(i, x) = 1 iff ψ i (x) is defined.…”

“…Note that Wiehagen and Zeugmann (1992) and Wiehagen (1992) dealt already with the inconsistency phenomenon in inductive inference and in exact learning in polynomial time. Furthermore, part of the present paper has been published in Wiehagen and Zeugmann (1994).…”

Consistency Conditions for Inductive Inference of Recursive Functions

“…The reader is encouraged to consult e.g., Jain et al [11], Fulk [7], Freivalds, Kinber and Wiehagen [6] and Wiehagen and Zeugmann [22,23] for further investigations concerning consistent and inconsistent learning.…”

A Discontinuity in Pattern Inference

“…In the sequel there has been a variety of profound studies (e.g. in [LW91], [WZ94], [RZ98], and many more) on the complexity of learning algorithms, consequences of different input data, efficient strategies for subclasses, and so on. Regarding E-pattern languages, however, appropriate approaches presumably need to be more sophisticated and therefore progress has been rather scarce.…”