Extended HMM and Ranking Models for Chinese Spelling Correction

Xiong, Jinhua; Zhang, Qiao; Hou, Jianpeng; Wang, Qianbo; Wang, Yuanzhuo; Cheng, Xueqi

doi:10.3115/v1/w14-6821

Cited by 15 publications

(13 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conversely, if the matched pair made by the character and the character preceding it or following it is commonly seen in the text corpus, then that character's degree of erroneousness is very low here. Xiong et al (2014) proposed using the Hidden Markov Model (HMM) as the basis for a model to detect and correct erroneous characters. This method presupposes that unknown erroneous characters exist in the sentence, and seeks out each character's substitute character by means of phonetic writing (pinyin) and the Cangjie input code using Bayes' rule as its basis.…”

Section: Related Workmentioning

confidence: 99%

Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing

Yu¹,

Zhi-fang²,

Zhang³

et al. 2015

View full text Add to dashboard Cite

Discourse in linguistics refers to a unit of language longer than a single sentence. It has not been well studied in the research community of computational linguistics, but it has attracted more and more attention in very recent years. This talk consists of two parts, i.e., discourse and machine translation. We will first give an overview about discourse and review the research state-of-the-art of discourse from both linguistics and computational viewpoints, and then discuss how machine translation can benefit from discourse-level information. Finally, we conclude the talk with some future direction discussions. BiographyZHANG Min: a distinguished professor and vice dean of the school of computer science and technology, director of the research Institute for Human Language Technology at Soochow University (China), received his Ph.D. degree in computer science from Harbin Institute of Technology (China) in 1997. He has studied and worked oversea in industry and academy at South Korea and Singapore since 1997 to 2013. His current research interests include machine translation and natural language processing. He has co-authored 2 Springer books and more than 130 papers in leading journals and conferences, and co-edited 13 books published by Springer and IEEE. He is an associate editor of IEEE T-ASLP (2015-2017).SONG Rou: a professor and Ph.D. supervisor at Applied Linguistics and Computer Application in Beijing Language and Culture University, received his Bachelor degree in mathematics and mechanics from Beijing University in 1968 and his mater Master degree in computer science from Beijing University in 1981. He has been working on Chinese Information Processing study for tens of years as the PIs of more than 10 national-level projects with the research focuses on discourse analysis, Chinese word segmentation, Computer-aided proofreading, Chinese word attribute, Chinese Orthographic Computing and Chinese POS and so on. He has published more than 100 papers at leading journals and conferences in computer science and linguistics. He has developed and commercialized several softwares with two patents. He has received several awards from Beijing City and MOE, China. He has been appointed as guest professors in a few domestic and oversea universities and research institutes. v

show abstract

Section: Related Workmentioning

confidence: 99%

Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing

Yu¹,

Zhi-fang²,

Zhang³

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Our method combines different methods to improve performance. The main contributions compared with our previous work (Xiong et al, 2014) are:…”

Section: Introductionmentioning

confidence: 99%

HANSpeller++: A Unified Framework for Chinese Spelling Correction

Zhang¹,

Xiong²,

Hou³

et al. 2015

Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing

Self Cite

View full text Add to dashboard Cite

Increased interest in China from foreigners has led to a corresponding interest in the study of Chinese. However, the learning of Chinese by non-native speakers will encounter many difficulties, Chinese spelling check techniques for Chinese as a Foreign Language(CFL) learners is highly desirable. This paper presents our work on the SIGHAN-2015 Chinese Spelling Check task. The task focuses on spelling checking on Chinese essays written by CFL learners. We propose a unified framework called HANSpeller++ based on our previous HANSpeller for Chinese spelling correction. The framework consists of candidate generating,candidates re-ranking and final global decision making. Experiments show good performance on the test data of the task.

show abstract

“…This makes the model more robust since error detection is a nontrivial task for social media texts due to high number of slang, proper names (including colloquial) etc. By its architecture our model more resembles Xiong et al (2014), however, the set of features used differs significantly reflecting the difference between Chinese and Russian. As far as we know, our model is one of the first HMM-based systems used for spelling correction of a morphologically rich language.…”

Section: Previous Workmentioning

confidence: 99%

Spelling Correction for Morphologically Rich Language: a Case Study of Russian

Sorokin¹

2017

Proceedings of the 6th Workshop on Balto-Slavic Natural Language Processing

View full text Add to dashboard Cite

We present an algorithm for automatic correction of spelling errors on the sentence level, which uses noisy channel model and feature-based reranking of hypotheses. Our system is designed for Russian and clearly outperforms the winner of SpellRuEval-2016 competition. We show that language model size has the greatest influence on spelling correction quality. We also experiment with different types of features and show that morphological and semantic information also improves the accuracy of spellchecking.

show abstract

Extended HMM and Ranking Models for Chinese Spelling Correction

Cited by 15 publications

References 6 publications

Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing

Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing

HANSpeller++: A Unified Framework for Chinese Spelling Correction

Spelling Correction for Morphologically Rich Language: a Case Study of Russian

Contact Info

Product

Resources

About