Compilation Methods of Minimal Acyclic Finite-State Automata for Large Dictionaries

Otero

Implementation and Application of Automata

2006

Abstract. The paper introduces a robust spelling correction technique to deal with ill-formed input strings, including unknown parts of unknown length. In contrast to previous works, we derive profit from a finer dynamic programming construction, which takes advantage of the underlying grammatical structure, leading to an improved computational behavior and error repair quality. The formal description applies a deductive approach in order to simplify this task, separating it from the interpretation strategy, and including cut-off facilities.

Section: Resultsmentioning

confidence: 99%

Robust Spelling Correction

Otero

Implementation and Application of Automata

2006

“…We generate from G a numbered minimal acyclic finite automaton for the language L(G). In practice, we choose a device [4] generated by Galena [3]. A fa is a 5-tuple A = (Q, Σ, δ, q 0 , Q f ) where: Q is the set of states, Σ the set of input symbols, δ is a function of Q × Σ into 2 Q defining the transitions of the automaton, q 0 the initial state and Q f the set of final states.…”

Section: The Error Repair Modelmentioning

confidence: 99%

“…We choose to work with a lexicon for Galician built from Galena [3], which includes 304.331 different words, to illustrate this aspect. The lexicon is recognized by a fa containing 16.837 states connected by 43.446 transitions, whose entity we consider sufficient for our purposes.…”

Section: The System At Workmentioning

confidence: 99%

Automatic Spelling Correction in Galician

Advances in Natural Language Processing

Otero

Barcala

et al. 2004

Abstract. We describe a proposal on spelling correction intended to be applied on Galician, a Romance language. Our aim is to put into evidence the flexibility of a novelty technique that provides a quality equivalent to global strategies, but with a significantly minor computational cost. To do it, we take advantage of the grammatical background present in the recognizer, which allows us to dynamically gather information to the right and to the left of the point at which the recognition halts in a word, as long as this information could be considered as relevant for the repair process. The experimental tests prove the validity of our approach in relation to previous ones, focusing on both performance and costs.

Soft Computing in Web Information Retrieval

“…The main reasons for compressing a very large dictionary of words into a finite-state automaton are that its representation of the set of words is compact, and that the process of looking up a word in the dictionary is proportional to the length of the word, and therefore very fast [7]. Of particular interest for natural language processing applications are minimal acyclic finite-state automata, which recognize finite sets of words, and which can be constructed in various ways [15,5]. The aim of the present work was to build a general architecture to handle a large Spanish dictionary of synonyms [2].…”

Section: A Computational View Of Synonymymentioning

confidence: 99%

“…To complete this model, we only need the implementation of the functions Word to Index and Index to Word. Both functions operate over a special type of automata, the numbered minimal acyclic finite-state automata described in [5], allowing us to efficiently perform perfect hashing between numbers and words.…”

Section: General Architecture Of An Electronic Dictionary Of Synonymsmentioning

confidence: 99%

Access to a Large Dictionary of Spanish Synonyms: A Tool for Fuzzy Information Retrieval

Sobrino-Cerdeiriña¹,

Fernández-Lanza²,

Graña-Gil³

Summary. We start by analyzing the role of imprecision in information retrieval in the Web, some theoretical contributions for managing this problem and its presence in search engines, with special emphasis on the use of thesaurus in order to increase the number and relevance of the documents retrieved. We then present a general architecture for implementing large dictionaries in natural language processing applications which is able to store a considerable amount of data relating to the words contained in these dictionaries. In this modelling, efficient access to this information is guaranteed by the use of minimal deterministic acyclic finite-state automata. In addition, we implement a Spanish dictionary of synonyms and illustrate how our general model helps to transform the original dictionary into a computational framework capable of representing semantic relations between words. This process allows us to define synonymy as a gradual relation, which makes the final tool more suitable for word sense disambiguation tasks or for information retrieval applications than other traditional approaches. Moreover, our electronic dictionary, called Fdsa, will be freely available very soon for stand-alone use.