Conversational Image Editing: Incremental Intent Identification in a New Dialogue Task

Manuvinakurike, Ramesh; Bui, Trung; Chang, Walter; Georgila, Kallirroi

doi:10.18653/v1/w18-5033

Cited by 20 publications

(11 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The best accuracy reported on the aforementioned conversational image editing dataset was 74% on intent classification, ignoring actual attribute values (Manuvinakurike et al, 2018a). This result is not directly comparable to the best accuracy 61.3% on our dataset due to the difference in accuracy definition.…”

Section: Comparison To Performance On Image Editingcontrasting

confidence: 60%

ChartDialogs: Plotting from Natural Language Instructions

Shao¹,

Nakashole

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

This paper presents the problem of conversational plotting agents that carry out plotting actions from natural language instructions. To facilitate the development of such agents, we introduce CHARTDIALOGS, a new multi-turn dialog dataset, covering a popular plotting library, matplotlib. The dataset contains over 15, 000 dialog turns from 3, 200 dialogs covering the majority of matplotlib plot types. Extensive experiments show the bestperforming method achieving 61% plotting accuracy, demonstrating that the dataset presents a non-trivial challenge for future research on this task.

show abstract

Section: Comparison To Performance On Image Editingcontrasting

confidence: 60%

ChartDialogs: Plotting from Natural Language Instructions

Shao¹,

Nakashole

2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…In the literature, different types of models have been applied to the task of intent classifi-cation for incremental NLU (e.g. DeVault et al, 2009;Manuvinakurike et al, 2018;Constantin et al, 2019;Coman et al, 2019;Madureira and Schlangen, 2020, inter alia). A typical approach is to segment complete utterances into increasingly longer partial utterances.…”

Section: Previous Workmentioning

confidence: 99%

Not So Fast, Classifier – Accuracy and Entropy Reduction in Incremental Intent Classification

Hrycyk¹,

Zarcone²,

Hahn³

2021

Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI

View full text Add to dashboard Cite

Incremental intent classification requires the assignment of intent labels to partial utterances. However, partial utterances do not necessarily contain enough information to be mapped to the intent class of their complete utterance (correctly and with a certain degree of confidence). Using the final interpretation as the ground truth to measure a classifier's accuracy during intent classification of partial utterances is thus problematic. We release in-CLINC, a dataset of partial and full utterances with human annotations of plausible intent labels for different portions of each utterance, as an upper (human) baseline for incremental intent classification. We analyse the incremental annotations and propose entropy reduction as a measure of human annotators' convergence on an interpretation (i.e. intent label). We argue that, when the annotators do not converge to one or a few possible interpretations and yet the classifier already identifies the final intent class early on, it is a sign of overfitting that can be ascribed to artefacts in the dataset.

show abstract

“…Vision and dialogue is an emerging topic of intersection field between computer vision and natural language processing. Conversational image editing system research [17], [18] attempts to understand the user utterance and identify the user's intention in an interactive image editing task using existing image editing software such as Adobe Photoshop and OpenCV. Our proposed method has the same motivation to identify the user's intention; however, our editing system is based on image generative models.…”

Section: Related Work a Vision And Dialoguementioning

confidence: 99%

An Interactive Image Editing System Using an Uncertainty-Based Confirmation Strategy

et al. 2020

Self Cite

View full text Add to dashboard Cite

We propose an interactive image editing system that has a confirmation dialogue strategy using an entropy-based uncertainty calculation on its generated images with Deep Convolutional Generative Adversarial Networks (DCGAN). DCGAN is an image generative model that learns an image manifold of a given dataset and enables continuous change of an image. Our proposed image editing system combines DCGAN with a natural language interface that accepts image editing requests in natural language. Although such a system is helpful for human users, it often faces uncertain requests to generate acceptable images. A promising approach to solve this problem is introducing a dialogue process that shows multiple candidates and confirms the user's intention. However, confirming every editing request creates redundant dialogues. To achieve more efficient dialogues, we propose an entropy-based dialogue strategy that decides when the system should confirm, and enables effective image editing through a dialogue that reduces redundant confirmations. We conducted image editing dialogue experiments using an avatar face illustration dataset for editing by natural language requests. Through quantitative and qualitative analysis, our results show that our entropy-based confirmation strategy achieved an effective dialogue by generating images desired by users. INDEX TERMS Confirmation, generative adversarial networks, image editing, natural language interface.

show abstract

Conversational Image Editing: Incremental Intent Identification in a New Dialogue Task

Cited by 20 publications

References 35 publications

ChartDialogs: Plotting from Natural Language Instructions

ChartDialogs: Plotting from Natural Language Instructions

Not So Fast, Classifier – Accuracy and Entropy Reduction in Incremental Intent Classification

An Interactive Image Editing System Using an Uncertainty-Based Confirmation Strategy

Contact Info

Product

Resources

About