Modeling Content and Context with Deep Relational Learning

Pacheco, María Leonor; Goldwasser, Dan

doi:10.1162/tacl_a_00357

Cited by 19 publications

(18 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that this information is only used for pre-training. Other works using social information to analyze political bias (Li and Goldwasser, 2019;Nguyen et al, 2020;Pacheco and Goldwasser, 2021) augment the text with social information, however since this information can be difficult to obtain in real-time, we decided to investigate if it can be used as a distant supervision source for pre-training a language model. These pre-training tasks are then used for training a Multi-head Attention Network (MAN) which creates a bias-aware representation of the text.…”

Section: Adapted From Nytimes (Left)mentioning

confidence: 99%

Using Social and Linguistic Information to Adapt Pretrained Representations for Political Perspective Identification

Chang¹,

Goldwasser²

2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

Self Cite

View full text Add to dashboard Cite

Understanding the political perspective shaping the way events are discussed in the media is increasingly important due to the dramatic change in news distribution. With the advance in text classification models, the performance of political perspective detection is also improving rapidly. However, current deep learning based text models often require a large amount of supervised data for training, which can be very expensive to obtain for this task. Meanwhile, models pre-trained on the general source and task (e.g. BERT) lack the ability to focus on bias-related text span. In this paper, we propose a novel framework that pretrains the text model using signals from the rich social and linguistic context that is readily available, including entity mentions, news sharing, and frame indicators. The pre-trained models benefit from tasks related to bias detection and therefore are easier to train with the bias labels. We demonstrate the effectiveness of our proposed framework by experiments on two news bias datasets. The models with pre-training achieve significant improvement in performance and are capable of identifying the text span for bias better.

show abstract

Section: Adapted From Nytimes (Left)mentioning

confidence: 99%

Using Social and Linguistic Information to Adapt Pretrained Representations for Political Perspective Identification

Chang¹,

Goldwasser²

2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…The pseudo-code for the structured learning procedure can be observed in Algorithm 1. We implemented our models using DRaiL (Pacheco and Goldwasser, 2020), a declarative deep structured prediction framework built on PyTorch, and extended it to support our randomized inference procedures 1 .…”

Section: Learningmentioning

confidence: 99%

Randomized Deep Structured Prediction for Discourse-Level Processing

Widmoser¹,

Pacheco

Honorio

et al. 2021

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

Self Cite

View full text Add to dashboard Cite

Expressive text encoders such as RNNs and Transformer Networks have been at the center of NLP models in recent work. Most of the effort has focused on sentence-level tasks, capturing the dependencies between words in a single sentence, or pairs of sentences. However, certain tasks, such as argumentation mining, require accounting for longer texts and complicated structural dependencies between them. Deep structured prediction is a general framework to combine the complementary strengths of expressive neural encoders and structured inference for highly structured domains. Nevertheless, when the need arises to go beyond sentences, most work relies on combining the output scores of independently trained classifiers. One of the main reasons for this is that constrained inference comes at a high computational cost. In this paper, we explore the use of randomized inference to alleviate this concern and show that we can efficiently leverage deep structured prediction and expressive neural encoders for a set of tasks involving complicated argumentative structures.

show abstract

“…In addition to modeling the narrative structure in the embedding space, we add a symbolic inference procedure to capture structural dependencies in the output space for the StoryCommonsense task. To model these dependencies, we use DRaiL (Pacheco and Goldwasser, 2021), a neural-symbolic framework that allows us to define probabilistic logical rules on top of neural network potentials.…”

Section: Symbolic Inferencementioning

confidence: 99%

“…This way, all neural parameters are updated to optimize the global objective. Additional details can be found in (Pacheco and Goldwasser, 2021). To score weighted rules, we used feed-forward networks over the node embeddings obtained by the objectives outlined in Sec.…”

Section: Symbolic Inferencementioning

confidence: 99%

See 1 more Smart Citation

Modeling Human Mental States with an Entity-based Narrative Graph

Lee¹,

Pacheco²,

Goldwasser³

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

Understanding narrative text requires capturing characters' motivations, goals, and mental states. This paper proposes an Entity-based Narrative Graph (ENG) to model the internalstates of characters in a story. We explicitly model entities, their interactions and the context in which they appear, and learn rich representations for them. We experiment with different task-adaptive pre-training objectives, in-domain training, and symbolic inference to capture dependencies between different decisions in the output space. We evaluate our model on two narrative understanding tasks: predicting character mental states, and desire fulfillment, and conduct a qualitative analysis.

show abstract

Modeling Content and Context with Deep Relational Learning

Cited by 19 publications

References 35 publications

Using Social and Linguistic Information to Adapt Pretrained Representations for Political Perspective Identification

Using Social and Linguistic Information to Adapt Pretrained Representations for Political Perspective Identification

Randomized Deep Structured Prediction for Discourse-Level Processing

Modeling Human Mental States with an Entity-based Narrative Graph

Contact Info

Product

Resources

About