Learning better discourse representation for implicit discourse relation recognition via attention networks

Zhang, Biao; Xiong, Deyi; Su, Jinsong; Zhang, Min

doi:10.1016/j.neucom.2017.09.074

Cited by 16 publications

(6 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Injecting hierarchy information into prompts is also promising. For example, using top-level predictions to refine prompts of bottom levels can surpass soft prompts and hard prompts (Wang et al, 2022b). Nevertheless, how to employ LLMs to better involve hierarchy knowledge is still under investigation.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition

Chan¹,

Liu²,

Cheng³

et al. 2023

Findings of the Association for Computational Linguistics: ACL 2023

View full text Add to dashboard Cite

Implicit Discourse Relation Recognition (IDRR) is a sophisticated and challenging task to recognize the discourse relations between the arguments with the absence of discourse connectives. The sense labels for each discourse relation follow a hierarchical classification scheme in the annotation process (Prasad et al., 2008), forming a hierarchy structure. Most existing works do not well incorporate the hierarchy structure but focus on the syntax features and the prior knowledge of connectives in the manner of pure text classification. We argue that it is more effective to predict the paths inside the hierarchical tree (e.g., "Comparison -> Contrast -> however") rather than flat labels (e.g., Contrast) or connectives (e.g., however). We propose a prompt-based path prediction method to utilize the interactive information and intrinsic senses among the hierarchy in IDRR. This is the first work that injects such structure information into pre-trained language models via prompt tuning, and the performance of our solution shows significant and consistent improvement against competitive baselines.

show abstract

Section: Related Workmentioning

confidence: 99%

“…• GOLF (Jiang et al, 2022b): a global and local hierarchy-aware contrastive framework, to model and capture the information from these two kinds of hierarchies with the aid of contrastive learning.…”

Section: A3 Baseline Modelsmentioning

confidence: 99%

DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition

Chan¹,

Liu²,

Cheng³

et al. 2023

Findings of the Association for Computational Linguistics: ACL 2023

View full text Add to dashboard Cite

show abstract

“…For the IDRR task, it is also argued that different words in arguments contribute differently in learning argument representations and interactions. Some researchers have proposed to augment conventional neural models with attention mechanisms, such as the attention-based CNNs [98,167] and attention-based RNNs [12,21,58,83]. For example, Zhang et al [167] proposed a full attention model that combines an inner attention model to process internal argument information and an outer attention model to exploit external world knowledge.…”

Section: Attention Mechanismmentioning

confidence: 99%

“…Some researchers have proposed to augment conventional neural models with attention mechanisms, such as the attention-based CNNs [98,167] and attention-based RNNs [12,21,58,83]. For example, Zhang et al [167] proposed a full attention model that combines an inner attention model to process internal argument information and an outer attention model to exploit external world knowledge. In the inner attention model, they employed the SCNN [165] to encode each argument as its original representation, which is then used to compute the attention weight of each word by a score function.…”

Section: Attention Mechanismmentioning

confidence: 99%

A Survey of Implicit Discourse Relation Recognition

Wang¹,

Wang²

2022

Preprint

View full text Add to dashboard Cite

A discourse containing one or more sentences describes daily issues and events for people to communicate their thoughts and opinions. As sentences are normally consist of multiple text segments, correct understanding of the theme of a discourse should take into consideration of the relations in between text segments. Although sometimes a connective exists in raw texts for conveying relations, it is more often the cases that no connective exists in between two text segments but some implicit relation does exist in between them. The task of implicit discourse relation recognition (IDRR) is to detect implicit relation and classify its sense between two text segments without a connective. Indeed, the IDRR task is important to diverse downstream natural language processing tasks, such as text summarization, machine translation and so on. This article provides a comprehensive and up-to-date survey for the IDRR task. We first summarize the task definition and data sources widely used in the field. We categorize the main solution approaches for the IDRR task from the viewpoint of its development history. In each solution category, we present and analyze the most representative methods, including their origins, ideas, strengths and weaknesses. We also present performance comparisons for those solutions experimented on a public corpus with standard data processing procedures. Finally, we discuss future research directions for discourse relation analysis.

show abstract

“…Based on bidirectional recurrent neural networks, Bahdanau et al [20] added the attention mechanism to the model to encode and decode the sentence in machine translation. Zhang et al [21] examined inner attention mechanism and outer attention mechanism in discourse representation for implicit discourse relation recognition. The result showed a marvelous improvement on marco- F 1 point is 1.61%.…”

Section: Related Workmentioning

confidence: 99%

A Stacked BiLSTM Neural Network Based on Coattention Mechanism for Question Answering

Cai

Sitong

Yan

et al. 2019

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Deep learning is the crucial technology in intelligent question answering research tasks. Nowadays, extensive studies on question answering have been conducted by adopting the methods of deep learning. The challenge is that it not only requires an effective semantic understanding model to generate a textual representation but also needs the consideration of semantic interaction between questions and answers simultaneously. In this paper, we propose a stacked Bidirectional Long Short-Term Memory (BiLSTM) neural network based on the coattention mechanism to extract the interaction between questions and answers, combining cosine similarity and Euclidean distance to score the question and answer sentences. Experiments are tested and evaluated on publicly available Text REtrieval Conference (TREC) 8-13 dataset and Wiki-QA dataset. Experimental results confirm that the proposed model is efficient and particularly it achieves a higher mean average precision (MAR) of 0.7613 and mean reciprocal rank (MRR) of 0.8401 on the TREC dataset.

show abstract

Learning better discourse representation for implicit discourse relation recognition via attention networks

Cited by 16 publications

References 7 publications

DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition

DiscoPrompt: Path Prediction Prompt Tuning for Implicit Discourse Relation Recognition

A Survey of Implicit Discourse Relation Recognition

A Stacked BiLSTM Neural Network Based on Coattention Mechanism for Question Answering

Contact Info

Product

Resources

About