Neural MultiVoice Models for Expressing Novel Personalities in Dialog

Oraby, Shereen; Reed, Lena; Sharath, T S; Tandon, Shubhangi; Walker, Marilyn A.

doi:10.21437/interspeech.2018-2174

Cited by 10 publications

(13 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We first discuss the baselines from previous work on the same benchmarks. Interestingly, for Personality, our NOCON model gets a huge performance improvement of more than 11 points in BLEU (27.74 → 38.45) over results reported by Oraby et al (2018a). We note that while the underlying architecture behind our experiments is similar to the baseline described by Oraby et al (2018a), we experiment with different parameters and attention mechanisms.…”

Section: Semantic Qualitymentioning

confidence: 82%

“…Interestingly, for Personality, our NOCON model gets a huge performance improvement of more than 11 points in BLEU (27.74 → 38.45) over results reported by Oraby et al (2018a). We note that while the underlying architecture behind our experiments is similar to the baseline described by Oraby et al (2018a), we experiment with different parameters and attention mechanisms. and Oraby et al (2018b) also use an LSTM encoderdecoder model with attention, but they both implement their models using the TGen 7 (Dušek and Jurcicek, 2016) framework with its default model architecture.…”

Section: Semantic Qualitymentioning

confidence: 82%

See 1 more Smart Citation

Maximizing Stylistic Control and Semantic Accuracy in NLG: Personality Variation and Discourse Contrast

Harrison¹,

Reed²,

Oraby³

et al. 2019

Proceedings of the 1st Workshop on Discourse Structure in Neural NLG

Self Cite

View full text Add to dashboard Cite

Neural generation methods for task-oriented dialogue typically generate from a meaning representation that is populated using a database of domain information, such as a table of data describing a restaurant. While earlier work focused solely on the semantic fidelity of outputs, recent work has started to explore methods for controlling the style of the generated text while simultaneously achieving semantic accuracy. Here we experiment with two stylistic benchmark tasks, generating language that exhibits variation in personality, and generating discourse contrast. We report a huge performance improvement in both stylistic control and semantic accuracy over the state of the art on both of these benchmarks. We test several different models and show that putting stylistic conditioning in the decoder and eliminating the semantic re-ranker used in earlier models results in more than 15 points higher BLEU for Personality, with a reduction of semantic error to near zero. We also report an improvement from .75 to .81 in controlling contrast and a reduction in semantic error from 16% to 2%.

show abstract

Section: Semantic Qualitymentioning

confidence: 82%

Section: Semantic Qualitymentioning

confidence: 82%

Maximizing Stylistic Control and Semantic Accuracy in NLG: Personality Variation and Discourse Contrast

Harrison¹,

Reed²,

Oraby³

et al. 2019

Proceedings of the 1st Workshop on Discourse Structure in Neural NLG

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are many other possibilities with both the WikiData knowledge graph RGs and with corpora such as Viggo for tuning and testing regimes that we have not yet experimented with or fully evaluated. We also plan to carry out future experiments on a number of other challenging problems for NLG [36,28,29,16]. 10…”

Section: Discussionmentioning

confidence: 99%

Jurassic is (almost) All You Need: Few-Shot Meaning-to-Text Generation for Open-Domain Dialogue

Reed¹,

Li²,

Ramirez³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

One challenge with open-domain dialogue systems is the need to produce high-quality responses on any topic. We aim to improve the quality and coverage of Athena, an Alexa Prize dialogue system. We utilize Athena's response generators (RGs) to create training data for two new neural Meaning-to-Text RGs, Athena-GPT-Neo and Athena-Jurassic, for the movies, music, TV, sports, and video game domains. We conduct few-shot experiments, both within and cross-domain, with different tuning set sizes (2, 3, 10), prompt formats, and meaning representations (MRs) for sets of WikiData KG triples, and dialogue acts with 14 possible attribute combinations. Our evaluation uses and human evaluation metrics, and shows that with 10-shot tuning, Athena-Jurassic's performance is significantly better for coherence and semantic accuracy. Experiments with 2-shot tuning on completely novel MRs results in a huge performance drop for Athena-GPT-Neo, whose semantic accuracy falls to 0.41, and whose untrue hallucination rate increases to 12%. Experiments with dialogue acts for video games show that with 10-shot tuning, both models learn to control dialogue acts, but Athena-Jurassic has significantly higher coherence, and only 4% untrue hallucinations. Our results suggest that Athena-Jurassic can reliably produce outputs of high-quality for live systems with real users. To our knowledge, these are the first results demonstrating that few-shot tuning on a massive language model can create NLGs that generalize to new domains, and produce high-quality, semantically-controlled, conversational responses directly from MRs and KG triples.

show abstract

“…We find that while the vanilla model makes the fewest semantic errors, the outputs loses any distinctive stylistic variation. With Model Context, however, we are able to achieve our goal: we can both produce stylistically varied outputs that correlate with the required personalities, and preserve semantic fidelity with notably few errors [Oraby et al, 2018b, Oraby et al, 2018a.…”

Section: Extravertmentioning

confidence: 99%

Stylistic Control for Neural Natural Language Generation

Oraby¹

2022

Companion Proceedings of the Web Conference 2022

Self Cite

View full text Add to dashboard Cite

Stylistic Control for Neural Natural Language Generation by Shereen M. OrabyNeural models for generating text from structured representations of meaning have recently gained popularity in the natural language generation (NLG) community.Instead of using a traditional NLG pipeline involving separate modules for sentence planning and surface realization, neural models combine these steps into a single endto-end framework. This new paradigm allows for low effort data-driven generation, but makes it very unclear how to control model output and produce the required semantics with the desired syntactic or stylistic constructions for a given application. This thesis takes on the task of learning to control neural natural language generation systems, with the goal of producing natural language outputs that are both semantically correct and stylistically varied. We tackle three critical bottlenecks of neural NLG: how to introduce a mechanism to produce style with neural generators, how to systematically acquire massive amounts of data required for training them, and how to jointly control semantic and stylistic choices, to allow for more diverse model outputs. We address the style bottleneck by experimenting with different methods for supervision in neural models with a synthetic dataset that we build (PersonageNLG), showing that we can produce diverse sentence planning operations in our model outputs. We address the data bottleneck by using freely available review data to create a massive, highly descriptive, and stylistically diverse corpus for training neural generators (YelpNLG), instead of relying on crowdsourcx ing. We address the control bottleneck by constructing stylistically rich meaning representations derived from review text based on parse information and freelyavailable ontologies, providing different forms of supervision to our neural models, and allowing us to produce outputs exhibiting a rich array of stylistic variation from semantically-grounded inputs.We show that by controlling the nature of our input data and how it is represented in our models, we can control a model's ability to produce a required style without sacrificing its ability to produce fluent outputs that express the required content. Our data and experiments introduce novel methods for producing stylistic variation within a neural natural language generation pipeline, and are generalizable to new domains and style choices.xi My PhD path has not been one that I have walked alone, and I would like to thank the many people that have guided me steadily along the way.My greatest thanks go to my advisor, Lyn Walker, for her genuinely tireless support throughout my time as a PhD student. I have seen first-hand that she is truly as enthusiastic about language and research as she is brilliant: and I must say that her enthusiasm is contagious. She has pushed me to be more critical of my findings, more principled in my approaches, and above all, to remain curious. She has been, and will continue to be, an inspiration to me both in and out of the res...

show abstract

Neural MultiVoice Models for Expressing Novel Personalities in Dialog

Cited by 10 publications

References 27 publications

Maximizing Stylistic Control and Semantic Accuracy in NLG: Personality Variation and Discourse Contrast

Maximizing Stylistic Control and Semantic Accuracy in NLG: Personality Variation and Discourse Contrast

Jurassic is (almost) All You Need: Few-Shot Meaning-to-Text Generation for Open-Domain Dialogue

Stylistic Control for Neural Natural Language Generation

Contact Info

Product

Resources

About