When Less Is More: Using Less Context Information to Generate Better Utterances in Group Conversations

Zhang, Haisong; Chan, Zhangming; Yan, Shuicheng; Zhao, Dongyan; Yan, Rui

doi:10.1007/978-3-319-99495-6_7

Cited by 7 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Zhang et al (2018a), and tackle response generation, taking in previous utterances as input and the next utterance as output also specifically include the responding speaker and target addressee in the inputs and outputs). Zhang et al (2018a) report the BLEU-n (n based on n-grams, n = 1, 2, 3, 4) and METEOR (Banerjee and Lavie, 2005) scores (mentioning that the evaluation could be supplemented); report BLEU, ROUGE , noun mentions, and length of generated response, along with limited human evaluations for fluency, consistency, and informativeness; and report BLEU-n (n = 1, 2, 3, 4), METEOR, ROUGE-L (L for longest common subsequence), along with human evaluations for fluency, grammaticality, and rationality. Qiu et al (2020) focus on the dialogue thread structures which are utilized in , utilizing structured attention with Variational RNN, reporting the same automatic metrics BLEU-n (n = 1, 2, 3, 4), METEOR, ROUGE-L (L for longest common subsequence).…”

Section: Response Generationmentioning

confidence: 99%

Proceedings of the 15th International Conference on Natural Language Generation

2022

View full text Add to dashboard Cite

Bio: Margaret Mitchell is a researcher working on Ethical AI, currently focused on the ins and outs of ethics-informed AI development in tech. She has published over 50 papers on natural language generation, assistive technology, computer vision, and AI ethics, and holds multiple patents in the areas of conversation generation and sentiment classification. She currently works at Hugging Face driving forward work in the ML development ecosystem, ML data governance, AI evaluation, and AI ethics. She previously worked at Google AI as a Staff Research Scientist, where she founded and co-led Google's Ethical AI group, focused on foundational AI ethics research and operationalizing AI ethics Googleinternally. Before joining Google, she was a researcher at Microsoft Research, focused on computer vision-to-language generation; and was a postdoc at Johns Hopkins, focused on Bayesian modeling and information extraction. She holds a PhD in Computer Science from the University of Aberdeen and a Master's in computational linguistics from the University of Washington. While earning her degrees, she also worked from 2005-2012 on machine learning, neurological disorders, and assistive technology at Oregon Health and Science University. She has spearheaded a number of workshops and initiatives at the intersections of diversity, inclusion, computer science, and ethics. Her work has received awards from Secretary of Defense Ash Carter and the American Foundation for the Blind, and has been implemented by multiple technology companies. She likes gardening, dogs, and cats.

show abstract

Section: Response Generationmentioning

confidence: 99%

Proceedings of the 15th International Conference on Natural Language Generation

2022

View full text Add to dashboard Cite

show abstract

“…Zhang et al (2018a), and tackle response generation, taking in previous utterances as input and the next utterance as output also specifically include the responding speaker and target addressee in the inputs and outputs). Zhang et al (2018a) report the BLEU-n (n based on n-grams, n = 1, 2, 3, 4) and METEOR (Banerjee and Lavie, 2005) scores (mentioning that the evaluation could be supplemented); report BLEU, ROUGE (Lin, 2004), noun mentions, and length of generated response, along with limited human evaluations for fluency, consistency, and informativeness; and report BLEU-n (n = 1, 2, 3, 4), METEOR, ROUGE-L (L for longest common subsequence), along with human evaluations for fluency, grammaticality, and rationality. Qiu et al (2020) focus on the dialogue thread structures which are utilized in , utilizing structured attention with Variational RNN, reporting the same automatic metrics BLEU-n (n = 1, 2, 3, 4), METEOR, ROUGE-L (L for longest common subsequence).…”

Section: Response Generationmentioning

confidence: 99%

Towards Evaluation of Multi-party Dialogue Systems

Mahajan,

Santhanam,

Shaikh

2022

Proceedings of the 15th International Conference on Natural Language Generation

View full text Add to dashboard Cite

Recent research in the field of conversational AI has emphasized the need for standardization of the metrics used in evaluation. In this work, we focus on evaluation methods used for multi-party dialogue systems. We present an expanded taxonomy focusing on multi-party dialogue based on the need for evaluation dimensions that address challenges associated with the presence of multiple participants. We also survey the evaluation metrics utilized in current multi-party dialogue research, and present our findings with regards to inconsistencies within existing work. Furthermore, we discuss the subsequent need to have more consistent evaluation methodologies and benchmarks. We motivate how consistency will contribute towards a better understanding of progress in the field of multi-party dialogue systems.

show abstract

“…如前文所述, 可以对复杂的多方对话历史进行结构分离, 进而更有针对性地进行回复. Zhang 等[31] 将多方对话的历史消息话语按照 "@" 回复关系, 组织成一个树形结构, 然后将树形结构分割成多个序列组合, 进而使用说话对象所在分支的话语进行回复生成. 文献[31] 实验结果表明使用较少的上下文信息来排除无关分支的话语, 可以产生更好的结果.…”

unclassified

“…Zhang 等[31] 将多方对话的历史消息话语按照 "@" 回复关系, 组织成一个树形结构, 然后将树形结构分割成多个序列组合, 进而使用说话对象所在分支的话语进行回复生成. 文献[31] 实验结果表明使用较少的上下文信息来排除无关分支的话语, 可以产生更好的结果. 如表 2[7, 10, 11, 18, 22, 23, 30∼33] 中 TreeSplit 结果所示, 该模型在个别指标上远优于对完整对话历史进行建模的模型, 但该方法需要数据集中有显式的说话对象标签.…”

unclassified

A survey of multi-party dialogue research based on deep learning

Zhang¹,

Zhang²,

Liu³

2021

Sci. Sin.-Inf.

View full text Add to dashboard Cite

In recent years, with the extensive application of deep learning technology, breakthroughs have been made in the study of human-computer dialogue. However, most of the current human-machine dialogue systems are designed under the assumption that both parties are involved, and the research and application of more challenging multi-party human-machine dialogues are not yet mature. Based on the field of natural language processing, this paper will review the research progress of multi-party dialogue based on deep learning in recent years. First, from the perspective of human-machine dialogue, we sort out the key problems and existing solutions of the multi-party dialogue system; then, we introduce other natural language processing tasks based on multi-party dialogue; afterwards, we summarize the existing multi-party dialogue research dataset and make a comparative analysis of limitations on the existing dataset; Finally, we look forward to the future development trend of multi-party dialogue research.

show abstract

When Less Is More: Using Less Context Information to Generate Better Utterances in Group Conversations

Cited by 7 publications

References 26 publications

Proceedings of the 15th International Conference on Natural Language Generation

Proceedings of the 15th International Conference on Natural Language Generation

Towards Evaluation of Multi-party Dialogue Systems

A survey of multi-party dialogue research based on deep learning

Contact Info

Product

Resources

About