Stylistic Control for Neural Natural Language Generation

Abstract

Stylistic Control for Neural Natural Language Generation by Shereen M. OrabyNeural models for generating text from structured representations of meaning have recently gained popularity in the natural language generation (NLG) community.Instead of using a traditional NLG pipeline involving separate modules for sentence planning and surface realization, neural models combine these steps into a single endto-end framework. This new paradigm allows for low effort data-driven generation, but makes it very unclear how to control model output and produce the required semantics with the desired syntactic or stylistic constructions for a given application. This thesis takes on the task of learning to control neural natural language generation systems, with the goal of producing natural language outputs that are both semantically correct and stylistically varied. We tackle three critical bottlenecks of neural NLG: how to introduce a mechanism to produce style with neural generators, how to systematically acquire massive amounts of data required for training them, and how to jointly control semantic and stylistic choices, to allow for more diverse model outputs. We address the style bottleneck by experimenting with different methods for supervision in neural models with a synthetic dataset that we build (PersonageNLG), showing that we can produce diverse sentence planning operations in our model outputs. We address the data bottleneck by using freely available review data to create a massive, highly descriptive, and stylistically diverse corpus for training neural generators (YelpNLG), instead of relying on crowdsourcx ing. We address the control bottleneck by constructing stylistically rich meaning representations derived from review text based on parse information and freelyavailable ontologies, providing different forms of supervision to our neural models, and allowing us to produce outputs exhibiting a rich array of stylistic variation from semantically-grounded inputs.We show that by controlling the nature of our input data and how it is represented in our models, we can control a model's ability to produce a required style without sacrificing its ability to produce fluent outputs that express the required content. Our data and experiments introduce novel methods for producing stylistic variation within a neural natural language generation pipeline, and are generalizable to new domains and style choices.xi My PhD path has not been one that I have walked alone, and I would like to thank the many people that have guided me steadily along the way.My greatest thanks go to my advisor, Lyn Walker, for her genuinely tireless support throughout my time as a PhD student. I have seen first-hand that she is truly as enthusiastic about language and research as she is brilliant: and I must say that her enthusiasm is contagious. She has pushed me to be more critical of my findings, more principled in my approaches, and above all, to remain curious. She has been, and will continue to be, an inspiration to me both in and out of the res...