The Research of Knowledge-Based Chinese Segmentation Method

Abstract

Current Chinese segmenting method doesn’t consider grammar and semantics during the process of segmenting. So segmenting accuracy and speed are not good. Therefore, on the base of analyzing the peculiarity of Chinese, a knowledge-based Chinese segment method, KSM, is presented. It uses a dictionary of hierarchical structure to get all the re-segmented words and makes ambiguity processing easy and effective. It gets support not only from letters and words knowledge, but from grammar and semantics knowledge. It removes wrong segmentation by checking grammar and semantics. Also It can learn new words to improve the accuracy of segmenting.