Efficiency in the Identification in the Limit Learning Paradigm

Abstract

International audienceTwo different paths exist when one is interested in validating an idea for a learning algorithm. On the one hand, the practical approach consists in using the available data to test the quality of the learning algorithm (for instance the widely used cross-validation technique). On the other hand, a theoretical approach is possible by using a learning paradigm, which is an attempt to formalize what learning means. Such models provide a framework to study the behavior of learning algorithms and to formally establish their soundness. The most widely used learning paradigm in Grammatical Inference is the one known as identification in the limit. The original definition has been found lacking because no efficiency bound is required. It remains an open problem how to best incorporate a notion of efficiency and tractability in this framework. This chapter surveys the different refinements that have been developed and studied, and the challenges they face. Main results for each formalisation, along with comparisons, are provided

Robust Identification in the Limit from Incomplete Positive Data

“…In the case of identification in the limit from positive data, the answer in the general case is negative. 5 However, the results above -in particular strong robustness -help us understand the conditions sufficient for this situation to occur. In this way, this work helps take a step towards a compositional theory of language learning.…”

“…This data could be considered "deficient" if the distribution poorly represents the target concept. As discussed by Eyraud et al [5], PAC is not necessarily well-suited for the problem of learning formal languages. The approximate nature of correctness in PAC requires a notion of distance between formal languages to judge the quality of a proposed solution.…”

Subregular Complexity and Deep Learning

“…Additionally, the size of S + ∪ S − is also bounded by a polynomial with respect to the size of A. Readers are referred to de la Higuera (1997) and Eyraud et al (2016) for more details on analyses of efficiency in grammatical inference.…”

Designing and Learning Substitutable Plane Graph Grammars

“…However, this definition, initially designed for the learning of regular string languages, is already unsuitable as a model for context free string grammars [34], so one cannot expect this requirements to be fulfilled in the case of graph grammars. As it is beyond the scope of this paper to attempt to resolve this difficulty, we shall thus adopt this approach in this paper: for a complete discussion, the Reader is referred to this book chapter [11].…”