Universal Dependencies Parsing for Colloquial Singaporean English

Yang

2019

Int. J. Autom. Comput.

In generative dialog systems, learning representations for the dialog context is a crucial step in generating high quality responses. The dialog systems are required to capture useful and compact information from mutually dependent sentences such that the generation process can effectively attend to the central semantics. Unfortunately, existing methods may not effectively identify importance distributions for each lower position when computing an upper level feature, which may lead to the loss of information critical to the constitution of the final context representations. To address this issue, we propose a transfer learning based method named transfer hierarchical attention network (THAN). The THAN model can leverage useful prior knowledge from two related auxiliary tasks, i.e., keyword extraction and sentence entailment, to facilitate the dialog representation learning for the main dialog generation task. During the transfer process, the syntactic structure and semantic relationship from the auxiliary tasks are distilled to enhance both the wordlevel and sentence-level attention mechanisms for the dialog system. Empirically, extensive experiments on the Twitter Dialog Corpus and the PERSONA-CHAT dataset demonstrate the effectiveness of the proposed THAN model compared with the state-of-the-art methods.

Section: Deep Transfer Learning In Nlpmentioning

confidence: 99%

Section: Transfer Learning To Thanmentioning

confidence: 99%

Transfer Hierarchical Attention Network for Generative Dialog System

Yang

2019

Int. J. Autom. Comput.

“…Our method can be extended into neural stacking Wang et al (2017), by doing back-propagation training of the parser parameters during model training, which are leave for future work.…”

Section: Syntactic Featuresmentioning

confidence: 99%

End-to-End Neural Relation Extraction with Global Optimization

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

2017

Self Cite

162

122

Neural networks have shown promising results for relation extraction. State-ofthe-art models cast the task as an end-toend problem, solved incrementally using a local classifier. Yet previous work using statistical models have demonstrated that global optimization can achieve better performances compared to local classification. We build a globally optimized neural model for end-to-end relation extraction, proposing novel LSTM features in order to better learn context representations. In addition, we present a novel method to integrate syntactic information to facilitate global learning, yet requiring little background on syntactic grammars thus being easy to extend. Experimental results show that our proposed model is highly effective, achieving the best performances on two standard benchmarks.

“…Existing work on cross-lingual transfer can be classified into two categories. The first aims to train a dependency parsing model on source treebanks (McDonald et al, 2011;Guo et al, 2016a,b), or their adapted versions (Zhao et al, 2009;Tiedemann et al, 2014;Wang et al, 2017) in the target language. The second category, 1 https://github.com/zhangmeishan/CodeMixedTreebank namely annotation projection, aims to produce a set of large-scale training instances full of automatic dependencies by parsing parallel sentences (Hwa et al, 2005;Rasooli and Collins, 2015).…”

Section: Related Workmentioning

confidence: 99%

Cross-Lingual Dependency Parsing Using Code-Mixed TreeBank

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

2019

Self Cite

Treebank translation is a promising method for cross-lingual transfer of syntactic dependency knowledge. The basic idea is to map dependency arcs from a source treebank to its target translation according to word alignments. This method, however, can suffer from imperfect alignment between source and target words. To address this problem, we investigate syntactic transfer by code mixing, translating only confident words in a source treebank. Cross-lingual word embeddings are leveraged for transferring syntactic knowledge to the target from the resulting code-mixed treebank. Experiments on University Dependency Treebanks show that code-mixed treebanks are more effective than translated treebanks, giving highly competitive performances among cross-lingual parsing methods.