Research on the Method and System of Word Segmentation and POS Tagging for Ancient Chinese Medicine Literature

Fu, Xinping; Yuan, Ting; Li, Xuebo; Wang, Zhenguo; Zhou, Yang; Ju, Fangning; Li, Jintong; Chen, Xiaokang; Xiao-ming, Sang

doi:10.1109/bibm47256.2019.8983361

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Knowledge discovery from ancient TCM literature is still in its infancy and achieves only limited results due to the unique characteristics of ancient Chinese language relative to the modern one, as mentioned in Section Background, which always lead to a failure when applying the existing Natural Language Processing (NLP) tools on these ancient Chinese texts. Therefore, some authors turn to the fundamental tasks for ancient Chinese literature processing of TCM such as word segmentation and Part-of-Speech (POS) tagging in [18].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Extracting Disease-Specific Clinical Experiences from Ancient Literature of Traditional Chinese Medicine with Deep Learning

Lu¹,

Chen²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: Ancient literature of Traditional Chinese Medicine (TCM) contains massive clinical experiences which are important ingredient of TCM knowledge and valuable for TCM clinical practice of nowadays. However, it is difficult for TCM professionals to acquire such valuable experiences due to their massive volume and broad occurrence in the literature. Furthermore, different characteristics of ancient Chinese language from the modern one lead to additional challenges for analyzing the literature, regardless of how to perform the analyzing, manually or automatically with a software toolkit. Methods: In order to overcome the aforementioned challenges, we formalize a novel information extraction task for ancient literature of TCM, and the entities to be extracted are Disease-Specific Clinical Experiences (DSCEs) occurring in the literature. For the purpose, we have collected two corpora from ancient literature of TCM and annotated them manually with DSCEs occurrence information for the diseases pregnant abdominalgia and colporrhagia (妊娠腹痛及下血) and jaundice (黄疸) respectively. We further propose a deep learning and CRF-based algorithmic framework with character encoding of ancient Chinese, thus avoiding the special difficulty in word segmentation for ancient Chinese texts. We investigate the framework with different methods for contextual encoding of characters in a sentence, including CNN, Bi-LSTM and BERT, and diverse approaches to aggregate contextual information of characters into a sentence encoding, such as max-pooling and attention mechanism. After that all the encoded sentences in a section of the literature are passed through a Bi-LSTM-based sequence labelling model with CRF inference on its top to obtain an optimal label sequence for the sentences in the section. Results: We conduct a series of experiments on the two corpora to verify the effectiveness of our framework for the task, and evaluate its effectiveness with different metrics in two granularities of labelling, namely accuracy/F1-value in sentence-level labelling and precision/recall/F1-value in correct recognition of the whole DSCEs. Conclusion: The experimental results demonstrate that the deep learning and CRF-based framework with character encoding of ancient Chinese could achieve an accuracy of 80.40%±1.64% and an F1-value of 76.73%±1.59% for the sentence labelling, while for recognition of the whole DSCEs, it is able to obtain the recall of 44.97%±2.16% and the precision of 51.13%±2.64%, meaning that the framework is a promising baseline for further development of the novel information extraction task for TCM.

show abstract

Section: Related Workmentioning

confidence: 99%

“…For the DSCE-level labelling we employ precision, recall and F 1 -value to assess our framework. They are defined respectively in the formulas ( 16), ( 17) and (18).…”

Section: Datasets and Evaluation Metricsmentioning

confidence: 99%

Extracting Disease-Specific Clinical Experiences from Ancient Literature of Traditional Chinese Medicine with Deep Learning

Lu¹,

Chen²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…studied the ancient Chinese word segmentation and tagging technology by using Hidden Markov Model (HMM) Wang, Huang and He (2017). constructed a model of the POS tagging for the Pre-Qin literature based on the CRF model and combined feature template Fu et al (2019). developed a word segmentation system of ancient books by constructing a Thesaurus of Chinese medicine terminology and a special POS tagging method based on HMM and Java language.…”

mentioning

confidence: 99%

Review and prospect of research on ancient book information processing in China

Liu

Qin

Zhao

2021

MJLIS

View full text Add to dashboard Cite

The advent of the era of big data appears to be an unprecedented opportunity to the development of ancient book information processing in China. A comprehensive review of the research on ancient book information processing can help researchers understand the research progress and grasp the future research trend. Based on the life cycle of ancient book information processing, this paper reviews and summarizes the progress of digital resource construction, data mining, system construction and information service, and prospects the future research trend. Although progress has been made to some extent, the ancient book information processing is still in its infancy. It is expected that more researchers will pay attention to and engage in this field.

show abstract

Discovering the Research Issues of Classical Chinese Segmentation via Modern Chinese Segmentation System

Tsai

Wei²,

Tseng

2021

2021 9th International Conference on Orange Technology (ICOT)

View full text Add to dashboard Cite

Research on the Method and System of Word Segmentation and POS Tagging for Ancient Chinese Medicine Literature

Cited by 3 publications

References 0 publications

Extracting Disease-Specific Clinical Experiences from Ancient Literature of Traditional Chinese Medicine with Deep Learning

Extracting Disease-Specific Clinical Experiences from Ancient Literature of Traditional Chinese Medicine with Deep Learning

Review and prospect of research on ancient book information processing in China

Discovering the Research Issues of Classical Chinese Segmentation via Modern Chinese Segmentation System

Contact Info

Product

Resources

About