A Neural Probabilistic Structured-Prediction Method for Transition-Based Natural Language Processing

Zhou, Hao; Zhang, Yue; Cheng, Chuan; Huang, Shujian; Dai, Xinyu; Chen, Jiajun

doi:10.1613/jair.5259

Cited by 8 publications

(6 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The end pointer of the stack changed position as the stack of tree nodes could be pushed and popped. Zhou et al [145] integrated beam search and contrastive learning for better optimization.…”

Section: B Parsingmentioning

confidence: 99%

Recent Trends in Deep Learning Based Natural Language Processing [Review Article]

Young

Hazarika

Poria

et al. 2018

IEEE Comput. Intell. Mag.

2,819

1,307

View full text Add to dashboard Cite

Deep learning methods employ multiple processing layers to learn hierarchical representations of data, and have produced state-of-the-art results in many domains. Recently, a variety of model designs and methods have blossomed in the context of natural language processing (NLP). In this paper, we review significant deep learning related models and methods that have been employed for numerous NLP tasks and provide a walk-through of their evolution. We also summarize, compare and contrast the various models and put forward a detailed understanding of the past, present and future of deep learning in NLP.

show abstract

Section: B Parsingmentioning

confidence: 99%

Recent Trends in Deep Learning Based Natural Language Processing [Review Article]

Young

Hazarika

Poria

et al. 2018

IEEE Comput. Intell. Mag.

2,819

1,307

View full text Add to dashboard Cite

show abstract

“…Thus all features in the neural segmentation model are dense features, alleviating the feature sparsity problem naturally, and meanwhile it is free of feature engineering. For better comparison between discrete and neural features, the overall segmentation framework of the baseline is kept, which includes the incremental segmentation process, the beam-search decoder and the training process integrated with beam-search (Zhang & Clark, 2011;Zhou et al, 2017). In addition, the neural network scorer takes the similar feature sources as the baseline, which includes both the input character sequence and the partially constructed output word sequence.…”

Section: Transition-based Neural Modelmentioning

confidence: 99%

“…Since it is challenging to integrate word features to the CRF inference framework of the existing character-based methods, we take inspiration from word-based discrete segmentation model instead. In particular, we follow Zhang and Clark (2007), using a transition-based framework (Zhang & Clark, 2011;Zhou, Zhang, Cheng, Huang, Dai, & Chen, 2017) to segment a sentence incrementally from left to right, scoring partially segmented results by using both character-level and word-level features. We replace the discrete word and character features of Zhang and Clark (2007) with neural word and character representations, respectively, and change their linear model into a deep neural network.…”

Section: Introductionmentioning

confidence: 99%

Transition-Based Neural Word Segmentation Using Word-Level Features

Zhang¹,

Zhang²,

Fu³

2018

jair

Self Cite

View full text Add to dashboard Cite

Character-based and word-based methods are two different solutions for Chinese word segmentation, the former exploiting sequence labeling models over characters and the latter using word-level features. Neural models have been exploited for character-based Chinese word segmentation, giving high accuracies by making use of external character embeddings, yet requiring less feature engineering. In this paper, we study a neural model for word-based Chinese word segmentation, by replacing the manually-designed discrete features with neural features in a transition-based word segmentation framework. Experimental results demonstrate that word features lead to comparable performance to the best systems in the literature, and a further combination of discrete and neural features obtains top accuracies on several benchmarks.

show abstract

“…This is especially true with the advent of deep learning-based NLP. Currently, some of the best algorithms for part-of-speech tagging (Huang, Xu, & Yu, 2015), parsing (D. Chen & Manning, 2014;Dyer, Ballesteros, Ling, Matthews, & Smith, 2015;Zhou et al, 2017;Zhu, Zhang, Chen, Zhang, & Zhu, 2013), named entity recognition (Chiu & Nichols, 2016;Passos, Kumar, & McCallum, 2014), sentiment classification (Mikolov, Sutskever, Chen, Corrado, & Dean, 2013), machine translation (Koehn, Och, & Marcu, 2003), and contextual embeddings (Q. Chen, Zhu, Ling, Wei, & Jiang, 2016;Liu, Shen, Duh, & Gao, 2017) are done with deep learning-based NLP.…”

Section: Introductionmentioning

confidence: 99%

Natural Language Processing-Based Quantication of the Mental State of Psychiatric Patients

Mukherjee¹,

Yu²,

Won³

et al. 2020

Computational Psychiatry

View full text Add to dashboard Cite

Psychiatric practice routinely uses semistructured and/or unstructured free text to record the behavior and mental state of patients. Many of these data are unstructured, lack standardization, and are difficult to use for analysis. Thus, it is difficult to quantitatively analyze a patient's illness trajectory over time and his or her responsiveness to treatment, and it is also difficult to compare different patients quantitatively. In this article, experts in the field of psychiatry, along with machine learning models, have collaboratively transformed patient data available in status assessments generated by physicians into binary vector representations. Data from patients with mental health disorders collected within a real-world clinical setting from one of the largest behavioral electronic health record (EHR) systems in the United States have been used for generating these representations. The binary vector representation of these health records is shown to be useful in various clinical tasks, such as disease phenotyping, characterizing the suicidality of patients, and inferring diagnoses. To summarize, this approach can transform semistructured free-text summaries of patients' status assessments into a structured, quantifiable format, which enriches the data that reside within EHR systems. This allows for effective intra-and interpatient quantifications and comparisons, which are much needed in the field of mental health. With the aid of these binary representations, patients' mental states can be systematically tracked over time, as can their responses to medications at the individual and population levels.

show abstract

A Neural Probabilistic Structured-Prediction Method for Transition-Based Natural Language Processing

Cited by 8 publications

References 51 publications

Recent Trends in Deep Learning Based Natural Language Processing [Review Article]

Recent Trends in Deep Learning Based Natural Language Processing [Review Article]

Transition-Based Neural Word Segmentation Using Word-Level Features

Natural Language Processing-Based Quantication of the Mental State of Psychiatric Patients

Contact Info

Product

Resources

About