The POLUSA Dataset: 0.9M Political News Articles Balanced by Time and Outlet Popularity

Gebhard, Lukas; Hamborg, Felix

doi:10.1145/3383583.3398567

Cited by 14 publications

(11 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…NoConflict Team NoConflict used their model of protest event sentence classification from the winning submission of the English version of Task 1 Subtask 2. Their model is based on a RoBERTa (Liu et al, 2019) backbone with a second pretraining (Gururangan et al, 2020) stage done on the POLUSA (Gebhard and Hamborg, 2020) data set before finetuned on Subtask 2 data. For the NYT data set, they first filtered the articles based on the section name.…”

Section: Team Systemsmentioning

confidence: 99%

Discovering Black Lives Matter Events in the United States: Shared Task 3, CASE 2021

Giorgi¹,

Zavarella²,

Tanev³

et al. 2021

Proceedings of the 4th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events From Text (CAS

View full text Add to dashboard Cite

Evaluating the state-of-the-art event detection systems on determining spatio-temporal distribution of the events on the ground is performed unfrequently. But, the ability to both (1) extract events "in the wild" from text and (2) properly evaluate event detection systems has potential to support a wide variety of tasks such as monitoring the activity of sociopolitical movements, examining media coverage and public support of these movements, and informing policy decisions. Therefore, we study performance of the best event detection systems on detecting Black Lives Matter (BLM) events from tweets and news articles. The murder of George Floyd, an unarmed Black man, at the hands of police officers received global attention throughout the second half of 2020. Protests against police violence emerged worldwide and the BLM movement, which was once mostly regulated to the United States, was now seeing activity globally. This shared task asks participants to identify BLM related events from large unstructured data sources, using systems pretrained to extract socio-political events from text. We evaluate several metrics, assessing each system's ability to evolution of protest events both temporally and spatially. Results show that identifying daily protest counts is an easier task than classifying spatial and temporal protest trends simultaneously, with maximum performance of 0.745 (Spearman) and 0.210 (Pearson r), respectively. Additionally, all baselines and participant systems suffered from low recall (max.5.08), confirming the high impact of media sourcing in the modelling of protest movements.

show abstract

Section: Team Systemsmentioning

confidence: 99%

Discovering Black Lives Matter Events in the United States: Shared Task 3, CASE 2021

Giorgi¹,

Zavarella²,

Tanev³

et al. 2021

Proceedings of the 4th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events From Text (CAS

View full text Add to dashboard Cite

show abstract

“…Second Pretraining We start by conducting an additional round of pretraining of RoBERTa, initialized with the already pretrained weight, following Gururangan et al (2020). To this end, we pretrain on the POLUSA dataset (Gebhard and Hamborg, 2020) in an MLM setting with a masking probability of 0.15. We denote this pretraining step as Second Pretraining.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Due to time and resource constraint, however, we cannot gain access to articles from these outlets. Thus, we resort to POLUSA (Gebhard and Hamborg, 2020). While it is not from the same outlets, the fact that it only contains political news make it suitable for our purpose.…”

Section: A Appendixmentioning

confidence: 99%

“…Given the huge success of pretrained language models such as BERT (Devlin et al, 2019), RoBERTa (Liu et al, 2019), XLNet (Yang et al, 2019) and ELECTRA (Clark et al, 2020), we adopt RoBERTa as the backbone of our model. Inspired by the good result achieved through additional pretraining (Gururangan et al, 2020), we harness the POLUSA dataset (Gebhard and Hamborg, 2020) of political news articles to second-pretrain our model with a masked language modeling (MLM) objective. Further, we conduct a series of ablation studies to justify our design choices.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Team “NoConflict” at CASE 2021 Task 1: Pretraining for Sentence-Level Protest Event Detection

Hu¹,

Stoehr²

2021

Proceedings of the 4th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events From Text (CAS

View full text Add to dashboard Cite

An ever-increasing amount of text, in the form of social media posts and news articles, gives rise to new challenges and opportunities for the automatic extraction of socio-political events. In this paper, we present our submission 1 to the Shared Tasks on Socio-Political and Crisis Events Detection, Task 1, Multilingual Protest News Detection, Subtask 2, Event Sentence Classification, of CASE @ ACL-IJCNLP 2021. In our submission, we utilize the RoBERTa model with additional pretraining, and achieve the best F1 score of 0.8532 in event sentence classification in English and the second-best F1 score of 0.8700 in Portuguese via simple translation. We analyze the failure cases of our model. We also conduct an ablation study to show the effect of choosing the right pretrained language model, adding additional training data and data augmentation.

show abstract

“…For research and evaluation of the previously described system and its analysis methods, I currently use the datasets AllSides (Chen et al, 2018), NewsWCL50 (Hamborg et al, 2019c), and PO-LUSA (Gebhard and Hamborg, 2020), which have high diversity concerning outlets' political slant.…”

Section: System and Visualizationmentioning

confidence: 99%

Media Bias, the Social Sciences, and NLP: Automating Frame Analyses to Identify Bias by Word Choice and Labeling

Hamborg¹

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop

Self Cite

View full text Add to dashboard Cite

Media bias can strongly impact the public perception of topics reported in the news. A difficult to detect, yet powerful form of slanted news coverage is called bias by word choice and labeling (WCL). WCL bias can occur, for example, when journalists refer to the same semantic concept by using different terms that frame the concept differently and consequently may lead to different assessments by readers, such as the terms "freedom fighters" and "terrorists," or "gun rights" and "gun control." In this research project, I aim to devise methods that identify instances of WCL bias and estimate the frames they induce, e.g., not only is "terrorists" of negative polarity but also ascribes to aggression and fear. To achieve this, I plan to research methods using natural language processing and deep learning while employing models and using analysis concepts from the social sciences, where researchers have studied media bias for decades. The first results indicate the effectiveness of this interdisciplinary research approach. My vision is to devise a system that helps news readers to become aware of the differences in media coverage caused by bias.

show abstract

The POLUSA Dataset: 0.9M Political News Articles Balanced by Time and Outlet Popularity

Cited by 14 publications

References 8 publications

Discovering Black Lives Matter Events in the United States: Shared Task 3, CASE 2021

Discovering Black Lives Matter Events in the United States: Shared Task 3, CASE 2021

Team “NoConflict” at CASE 2021 Task 1: Pretraining for Sentence-Level Protest Event Detection

Media Bias, the Social Sciences, and NLP: Automating Frame Analyses to Identify Bias by Word Choice and Labeling

Contact Info

Product

Resources

About