Hooks in the Headline: Learning to Generate Headlines with Controlled Styles

Jin, Di; Jin, Zhijing; Zhou, Joey Tianyi; Orii, Lisa; Szolovits, Peter

doi:10.18653/v1/2020.acl-main.456

Cited by 38 publications

(43 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Summarizing documents into different styles are mainly studied on news articles, where one appends style codes as extra embeddings to the encoder (Fan et al, 2018), or connects separate decoders with a shared encoder (Zhang et al, 2018). Similar to our work, Jin et al (2020) leverage large pre-trained seq2seq models, but they modify model architecture by adding extra style-specific parameters. Nonetheless, existing work requires training new summarizers for different target styles or modifying the model structure.…”

Section: Related Workmentioning

confidence: 99%

“…Generating summaries with different language styles can benefit readers of varying literacy levels (Chandrasekaran et al, 2020) or interests (Jin et al, 2020). Significant progress has been made in abstractive summarization with large pre-trained Transformers (Dong et al, 2019;Lewis et al, 2020;Zhang et al, 2019;Raffel et al, 2019;Song et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Inference Time Style Control for Summarization

Cao¹,

Wang²

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

How to generate summaries of different styles without requiring corpora in the target styles, or training separate models? We present two novel methods that can be deployed during summary decoding on any pre-trained Transformer-based summarization model. (1) Decoder state adjustment instantly modifies decoder final states with externally trained style scorers, to iteratively refine the output against a target style. (2) Word unit prediction constrains the word usage to impose strong lexical control during generation. In experiments of summarizing with simplicity control, automatic evaluation and human judges both find our models producing outputs in simpler languages while still informative. We also generate news headlines with various ideological leanings, which can be distinguished by humans with a reasonable probability.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Inference Time Style Control for Summarization

Cao¹,

Wang²

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…Current benchmarks for style transfer focus on high-level style definitions such as transfer of sentiment (Shen et al, 2017;Lample et al, 2019;, politeness (Madaan et al, 2020), formality (Rao and Tetreault, 2018;Krishna et al, 2020), writing styles (Jhamtani et al, 2017;Syed et al, 2020;Jin et al, 2020) and some other styles (Kang and Hovy, 2019). However, these only focus on only high-level styles, unlike STYLEPTB.…”

Section: Related Workmentioning

confidence: 99%

“…2) SEQ2SEQ: A Seq2Seq model (Sutskever et al, 2014) with attention trained using MLE Jin et al, 2020).…”

Section: Baseline Modelsmentioning

confidence: 99%

StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer

Lyu¹,

Liang²,

Pham³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Text style transfer aims to controllably generate text with targeted stylistic changes while maintaining core meaning from the source sentence constant. Many of the existing style transfer benchmarks primarily focus on individual high-level semantic changes (e.g. positive to negative), which enable controllability at a high level but do not offer fine-grained control involving sentence structure, emphasis, and content of the sentence. In this paper, we introduce a large-scale benchmark, STYLEPTB, with (1) paired sentences undergoing 21 fine-grained stylistic changes spanning atomic lexical, syntactic, semantic, and thematic transfers of text, as well as (2) compositions of multiple transfers which allow modeling of fine-grained stylistic changes as building blocks for more complex, high-level transfers. By benchmarking existing methods on STYLEPTB, we find that they struggle to model fine-grained changes and have an even more difficult time composing multiple styles. As a result, STYLEPTB brings novel challenges that we hope will encourage future research in controllable text style transfer, compositional models, and learning disentangled representations. Solving these challenges would present important steps towards controllable text generation.

show abstract

“…News Headlines in NLP. Headlines are popular as a challenging source for generation tasks such as summarization (Rush et al, 2015), style transfer (Jin et al, 2020), and style-preserving translation (Joshi et al, 2013). Headlines have been leveraged to detect political bias (Gangula et al, 2019), clickbait and fake news phenomena (Bourgonje et al, 2017).…”

Section: Related Workmentioning

confidence: 99%

News Headline Grouping as a Challenging NLU Task

Laban¹,

Bandarkar²,

Hearst³

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Recent progress in Natural Language Understanding (NLU) has seen the latest models outperform human performance on many standard tasks. These impressive results have led the community to introspect on dataset limitations, and iterate on more nuanced challenges. In this paper, we introduce the task of HeadLine Grouping (HLG) and a corresponding dataset (HLGD) consisting of 20,056 pairs of news headlines, each labeled with a binary judgement as to whether the pair belongs within the same group. On HLGD, human annotators achieve high performance of around 0.9 F-1, while current state-of-the art Transformer models only reach 0.75 F-1, opening the path for further improvements. We further propose a novel unsupervised Headline Generator Swap model for the task of HeadLine Grouping that achieves within 3 F-1 of the best supervised model. Finally, we analyze highperforming models with consistency tests, and find that models are not consistent in their predictions, revealing modeling limits of current architectures.

show abstract

Hooks in the Headline: Learning to Generate Headlines with Controlled Styles

Cited by 38 publications

References 35 publications

Inference Time Style Control for Summarization

Inference Time Style Control for Summarization

StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer

News Headline Grouping as a Challenging NLU Task

Contact Info

Product

Resources

About