Metaheuristics for Natural Language Tagging

Abstract

Abstract. This work compares different metaheuristics techniques applied to an important problem in natural language: tagging. Tagging amounts to assigning to each word in a text one of its possible lexical categories (tags) according to the context in which the word is used (thus it is a disambiguation task). Specifically, we have applied a classic genetic algorithm (GA), a CHC algorithm, and a Simulated Annealing (SA). The aim of the work is to determine which one is the most accurate algorithm (GA, CHC or SA), which one is the most appropriate encoding for the problem (integer or binary) and also to study the impact of parallelism on each considered method. The work has been highly simplified by the use of MALLBA, a library of search techniques which provides generic optimization software skeletons able to run in sequential, LAN and WAN environments. Experiments show that the GA with the integer encoding provides the more accurate results. For the CHC algorithm, the best results are obtained with binary coding and a parallel implementation. SA provides less accurate results than any of the evolutionary algorithms.

How evolutionary algorithms are applied to statistical natural language processing

“…This work was refined later by including parallel execution of the GA for tagging, (Araujo et al 2004;Alba et al 2006), as well as a comparison with other heuristic search methods applied to the problem, such as CHC algorithm (a non-traditional GA with particular mechanisms to maintain diversity in the population), and simulated annealing (SA). Results obtained show that the GA is able to perform tagging with the same accuracy (96.41 for the Brown corpus and 97.32 for the Susanne corpus) as the Viterbi method (which is specific for this problem) with additional scenarios (other kinds of contexts) forbidden to other techniques.…”

An Approach to the POS Tagging Problem Using Genetic Algorithms

“…These approaches can also be divided by the type of information used to solve the problem, statistical information [3,4,5,6,7], and rule-based information [9]. Shortly, in the former, an evolutionary algorithm is used to assign the most likely tag to each word of a sentence, based on a context table, that basically has the same information that is used in the traditional probabilistic approaches.…”

“…In this case a genetic algorithm (GA) is used to evolve a set of transformations rules, that will be used to tag a text in much the same way as the tagger proposed by Brill. While in [3,4,5,6,7] the evolutionary algorithm is used to discover the best sequence of tags for the words of a sentence, using an information model based on statistical data, in [9] the evolutionary algorithm is used to evolve the information model, in the form of a set of transformation rules, that will be used to tag the words of a sentence.…”

“…The first group is based on statistical data concerning the different context possibilities for a word [2,3,4,5,6,7], while the second group is based on rules, normally designed by human experts, that capture the language properties [8,9,10].…”

Tagging with Disambiguation Rules - A New Evolutionary Approach to the Part-of-Speech Tagging Problem

“…More recently, several evolutionary approaches have been proposed to solve the tagging problem. These approaches can also be divided by the type of information used to solve the problem, statistical information (Araujo, 2002;Araujo, 2004;Araujo, 2007;Araujo et al, 2004;Alba et al, 2006), and rule-based information (Wilson and Heywood, 2005). Shortly, in the former, an evolutionary algorithm is used to assign the most likely tag to each word of a sentence, based on a context table, that basically has the same information that is used in the traditional probabilistic approaches.…”