Unsupervised Learning of the Morphology of a Natural Language

Goldsmith, John

doi:10.1162/089120101750300490

Cited by 495 publications

(395 citation statements)

References 6 publications

Supporting

Mentioning

389

Contrasting

Unclassified

Order By: Relevance

“…One facet of NL morphological structure commonly leveraged by morphology induction algorithms is that morphemes are recurrent building blocks of words. Brent et al (1995), Goldsmith (2001), and Creutz (2006) emphasize the building block nature of morphemes when they each use recurring word segments to efficiently encode a corpus. These approaches then hypothesize that those recurring segments which most efficiently encode a corpus are likely morphemes.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

ParaMor: Finding Paradigms across Morphology

Monson

Carbonell

Lavie

et al. 2008

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Our algorithm, ParaMor, fared well in Morpho Challenge 2007 (Kurimo et al., 2007), a peer operated competition pitting against one another algorithms designed to discover the morphological structure of natural languages from nothing more than raw text. ParaMor constructs sets of affixes closely mimicking the paradigms of a language, and, with these structures in hand, annotates word forms with morpheme boundaries. Of the four language tracks in Morpho Challenge 2007, we entered ParaMor in English and German. Morpho Challenge 2007 evaluated systems on their precision, recall, and balanced F 1 at identifying morphological processes, whether those processes mark derivational morphology or inflectional features. In English, ParaMor's balanced precision and recall outperform at F 1 an already sophisticated baseline induction algorithm, Morfessor (Creutz, 2006). ParaMor placed fourth in English overall. In German, ParaMor suffers from a low morpheme recall. But combining ParaMor's analyses with analyses from Morfessor results in a set of analyses that outperform either algorithm alone, and that place first in F 1 among all algorithms submitted

show abstract

Section: Related Workmentioning

confidence: 99%

“…The paradigm structure of NL morphology has also been previously leveraged. Goldsmith (2001) uses morphemes to efficiently encode a corpus, but he first groups morphemes into paradigm like structures he calls signatures. To date, the work that draws the most on paradigm structure is Snover (2002).…”

Section: Related Workmentioning

confidence: 99%

ParaMor: Finding Paradigms across Morphology

Monson

Carbonell

Lavie

et al. 2008

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…For the moment, for being able to perform a comparison with Linguistica system (J. Goldsmith, [3]), we made the experiments for suffixes only, though the algorithms permits simultaneous treatment of suffixes and prefixes.…”

Section: Resultsmentioning

confidence: 99%

“…The prevalent approach is presented in [3] and it is implemented in Linguistica system. Variations of this method are described in [4], [7], and [1].…”

Section: Introductionmentioning

confidence: 99%

Division of Spanish Words into Morphemes with a Genetic Algorithm

Gelbukh

Sidorov

Lara-Reyes

et al.

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract.We discuss an unsupervised technique for determining morpheme structure of words in an inflective language, with Spanish as a case study. For this, we use a global optimization (implemented with a genetic algorithm), while most of the previous works are based on heuristics calculated using conditional probabilities of word parts. Thus, we deal with complete space of solutions and do not reduce it with the risk to eliminate some correct solutions beforehand. Also, we are working at the derivative level as contrasted with the more traditional grammatical level interested only in flexions. The algorithm works as follows. The input data is a wordlist built on the base of a large dictionary or corpus in the given language and the output data is the same wordlist with each word divided into morphemes. First, we build a redundant list of all strings that might possibly be prefixes, suffixes, and stems of the words in the wordlist. Then, we detect possible paradigms in this set and filter out all items from the lists of possible prefixes and suffixes (though not stems) that do not participate in such paradigms. Finally, a subset of those lists of possible prefixes, stems, and suffixes is chosen using the genetic algorithm. The fitness function is based on the ideas of minimum length description, i.e. we choose the minimum number of elements that are necessary for covering all the words. The obtained subset is used for dividing the words from the wordlist. Algorithm parameters are presented. Preliminary evaluation of the experimental results for a dictionary of Spanish is given.

show abstract

“…2. Unsupervised Machine Learning Approach: Goldsmith (2000) developed an unsupervised learning automatic morphology tool called AutoMorphology. This system is advantageous because it could automatically learn the most common prefixes and suffixes from just a word-list.…”

Section: Arabic Morphologymentioning

confidence: 99%

Adapting Morphology for Arabic Information Retrieval*

Darwish

Oard

Text, Speech and Language Technology

View full text Add to dashboard Cite

Abstract:This chapter presents an adaptation of existing techniques in Arabic morphology by leveraging corpus statistics to make them suitable for Information Retrieval (IR). The adaptation resulted in the development of Sebawai, an shallow Arabic morphological analyzer, and Al-Stem, an Arabic light stemmer. Both were used to produce Arabic index terms for Arabic experimentation. Sebawai is concerned with generating possible roots and stems of a given Arabic word along with probability estimates of deriving the word from each of the possible roots. The probability estimates were used as a guide to determine which prefixes and suffixes should be used to build the light stemmer Al-Stem. The use of the Sebawai generated roots and stems as index terms along with the stems from Al-Stem are evaluated in an information retrieval application and the results are compared IntroductionDue to the morphological complexity of the Arabic language, Arabic morphology has become an integral part of many Arabic Information Retrieval (IR) and other natural language processing applications. Arabic words are divided into three types: noun, verb, and particle (Abdul-Al-Aal, 1987). Nouns and verbs are derived from a closed set of around 10,000 roots (Ibn Manzour, 2006). The roots are commonly three or four letters and are rarely five letters. Arabic nouns and verbs * All the experiments for this work were performed while the first author was at the University of Maryland, College Park.

show abstract

Unsupervised Learning of the Morphology of a Natural Language

Cited by 495 publications

References 6 publications

ParaMor: Finding Paradigms across Morphology

ParaMor: Finding Paradigms across Morphology

Division of Spanish Words into Morphemes with a Genetic Algorithm

Adapting Morphology for Arabic Information Retrieval*

Contact Info

Product

Resources

About