An Algorithm for Predicting the Relation between Lemmas and Corpus Size

Yang, Dan-Hee; Gómez, Pascual Cantos; Song, Mansuk Song

doi:10.4218/etrij.00.0100.0203

Cited by 8 publications

(5 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They concluded that the UMLS Metathesaurus‐based CLIR method was at least equivalent to, if not better than, multilingual‐dictionary‐based approaches. Yang, Gomez, and Song (2000) observed that there was no reliable guideline as to how large, machine readable corpora should be compiled to develop practical NLP software packages and/or complete dictionaries for humans and computational use. They proposed a new mathematical tool—a piecewise curve‐fitting algorithm—and suggested how to determine the tolerance error of the algorithm for good prediction, using a specific corpus.…”

Section: Machine Translation and Clirmentioning

confidence: 99%

Natural language processing

Chowdhury

2003

Annual Review Info Sci & Tec

698

242

View full text Add to dashboard Cite

Beginning with the basic issues of NLP, this chapter aims to chart the major research activities in this area since the last ARIST Chapter in 1996 (Haas, 1996), including: (i) natural language text processing systems - text summarization, information extraction, information retrieval, etc., including domain-specific applications; (ii) natural language interfaces; (iii) NLP in the context of www and digital libraries ; and (iv) evaluation of NLP systems

show abstract

Section: Machine Translation and Clirmentioning

confidence: 99%

Natural language processing

Chowdhury

2003

Annual Review Info Sci & Tec

698

242

View full text Add to dashboard Cite

show abstract

“…Previous approaches have dealt with syntactic term mismatches caused by noun phrases in three ways. The first approach has tried to normalize noun phrases by the statistical term cooccurrence method (Fagon 1989;Yun, Cho, and Rim 1997;Yang 2000). This method defines noun phrases as two or more single terms that occur frequently in close proximity to each other in the texts.…”

Section: Methods To Handle Syntactic Term Mismatchmentioning

confidence: 99%

Semantic‐Based Information Retrieval for Content Management and Security

Yun

Seo

2003

Computational Intelligence

View full text Add to dashboard Cite

An information retrieval system has to retrieve all and only those documents that are relevant to a user query, even if index terms and query terms are not matched exactly. However, term mismatches between index terms and query terms have been a serious obstacle to the enhancement of retrieval performance. In this article, we discuss automatic term normalization between words and phrases in text corpora and their application to a Korean information retrieval system. We perform three new types of term normalizations: transliterated word normalization, noun phrase normalization, and context-based term normalization. Transliterated words are normalized into equivalence classes by using contextual similarity to alleviate lexical term mismatches. Then, noun phrases are normalized into phrasal terms by segmenting compound nouns as well as normalizing noun phrases. Moreover, context-based terms are normalized by using a combination of mutual information and word context to establish word similarities. Next, unsupervised clustering is done by using the K-means algorithm and cooccurrence clusters are identified to alleviate semantic term mismatches. These term normalizations are used in both the indexing and the retrieval system. The experimental results show that our proposed system can alleviate three types of term mismatches and can also provide the appropriate similarity measurements. As a result, our system can improve the retrieval effectiveness of the information retrieval system.

show abstract

“…However, it is impractical to get all the necessary triphone instances [8]. For the prosodic aspect, each triphone should retain enough instances to cover possible prosodic variations appearing in the utterances as the number of necessary triphone instances may reach over several millions in the end.…”

Section: Design Of Recording Sentence Setmentioning

confidence: 99%

“…For the prosodic aspect, each triphone should retain enough instances to cover possible prosodic variations appearing in the utterances as the number of necessary triphone instances may reach over several millions in the end. However, it is impractical to get all the necessary triphone instances [8].…”

Section: Design Of Recording Sentence Setmentioning

confidence: 99%

Unit Generation Based on Phrase Break Strength and Pruning for Corpus-Based Text-to-Speech

Kim¹,

Lee²,

Hirose³

2001

ETRI J

View full text Add to dashboard Cite

This paper discusses two important issues of corpus‐based synthesis: synthesis unit generation based on phrase break strength information and pruning redundant synthesis unit instances. First, the new sentence set for recording was designed to make an efficient synthesis database, reflecting the characteristics of the Korean language. To obtain prosodic context sensitive units, we graded major prosodic phrases into 5 distinctive levels according to pause length and then discriminated intra‐word triphones using the levels. Using the synthesis unit with phrase break strength information, synthetic speech was generated and evaluated subjectively. Second, a new pruning method based on weighted vector quantization (WVQ) was proposed to eliminate redundant synthesis unit instances from the synthesis database. WVQ takes the relative importance of each instance into account when clustering similar instances using vector quantization (VQ) technique. The proposed method was compared with two conventional pruning methods through objective and subjective evaluations of synthetic speech quality: one to simply limit the maximum number of instances, and the other based on normal VQ‐based clustering. For the same reduction rate of instance number, the proposed method showed the best performance. The synthetic speech with reduction rate 45% had almost no perceptible degradation as compared to the synthetic speech without instance reduction.

show abstract

An Algorithm for Predicting the Relation between Lemmas and Corpus Size

Cited by 8 publications

References 3 publications

Natural language processing

Natural language processing

Semantic‐Based Information Retrieval for Content Management and Security

Unit Generation Based on Phrase Break Strength and Pruning for Corpus-Based Text-to-Speech

Contact Info

Product

Resources

About