Numberings Optimal for Learning

Abstract

This paper extends previous studies on learnability in non-acceptable numberings by considering the question: for which criteria which numberings are optimal, that is, for which numberings it holds that one can learn every learnable class using the given numbering as hypothesis space. Furthermore an effective version of optimality is studied as well. It is shown that the effectively optimal numberings for finite learning are just the acceptable numberings. In contrast to this, there are non-acceptable numberings which are optimal for finite learning and effectively optimal for explanatory, vacillatory and behaviourally correct learning. The numberings effectively optimal for explanatory learning are the K-acceptable numberings. A similar characterization is obtained for the numberings which are effectively optimal for vacillatory learning. Furthermore, it is studied which numberings are optimal for one and not for another criterion: among the criteria of finite, explanatory, vacillatory and behaviourally correct learning all separations can be obtained; however every numbering which is optimal for explanatory learning is also optimal for consistent learning.

Hypothesis spaces for learning

“…Furthermore, we consider whether a hypothesis space being optimal for a particular learning criterion implies it being optimal for some other learning criterion. Such studies were done by [25].…”

“…Theorem 13. [25] (a) For each I ∈ {TxtEx, TxtFin, TxtBc, TxtFex}, there are programming systems which are optimal but not algorithmically optimal for I. (b) For any two distinct I and J in {TxtEx, TxtFin, TxtBc, TxtFex}, there is a programming system which is optimal for I but not optimal for J.…”

“…In learning with additional information, in addition to a text for the language, a learner is also provided with an upper bound on a (code for) grammar (in the hypothesis space) for the target language [16,23]. It was shown in [25] that the Ke-programming systems are exactly those hypothesis spaces which are optimal for learning with additional information.…”

“…[25] A hypothesis space H = (H i ) i∈N of all r.e. sets is (a) algorithmically optimal for TxtFin-learning iff H is acceptable; (b) algorithmically optimal for TxtEx-learning iff H is K-acceptable; (c) algorithmically optimal for TxtFex-learning iff there is a limiting-computable function g such that, for all d, there is an e ≤ g(d) with H e = W d .…”

Hypothesis Spaces for Learning

“…It was shown in [25] that the Ke-programming systems are exactly those hypothesis spaces which are optimal for learning with additional information.…”