Integrated Learning of Dialog Strategies and Semantic Parsing

Padmakumar, Aishwarya; Thomason, Jesse; Mooney, Raymond J.

doi:10.18653/v1/e17-1052

Cited by 18 publications

(19 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this may lead to a non-stationary environment (e.g., changing state transition) from the perspective of the agent, making its training (e.g., error detector learning) unstable. In the context of dialog systems, Padmakumar et al (2017) suggests that this effect can be mitigated by jointly updating the dialog policy and the semantic parser batchwisely. We leave exploring this aspect in our task to future work.…”

Section: Discussionmentioning

confidence: 99%

Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study

Yao¹,

Su²,

Sun³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

As a promising paradigm, interactive semantic parsing has shown to improve both semantic parsing accuracy and user confidence in the results. In this paper, we propose a new, unified formulation of the interactive semantic parsing problem, where the goal is to design a modelbased intelligent agent. The agent maintains its own state as the current predicted semantic parse, decides whether and where human intervention is needed, and generates a clarification question in natural language. A key part of the agent is a world model: it takes a percept (either an initial question or subsequent feedback from the user) and transitions to a new state. We then propose a simple yet remarkably effective instantiation of our framework, demonstrated on two text-to-SQL datasets (WikiSQL and Spider) with different state-of-the-art base semantic parsers. Compared to an existing interactive semantic parsing approach that treats the base parser as a black box, our approach solicits less user feedback but yields higher run-time accuracy.

show abstract

Section: Discussionmentioning

confidence: 99%

Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study

Yao¹,

Su²,

Sun³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…", a behavior annotation prompt. A human-robot POMDP dialog policy could be learned, as in previous work (Padmakumar et al, 2017), to know when this kind of follow-up question is warranted.…”

Section: Discussionmentioning

confidence: 99%

Guiding Interaction Behaviors for Multi-modal Grounded Language Learning

Thomason

Sinapov

Mooney

2017

Proceedings of the First Workshop on Language Grounding for Robotics

Self Cite

View full text Add to dashboard Cite

Multi-modal grounded language learning connects language predicates to physical properties of objects in the world. Sensing with multiple modalities, such as audio, haptics, and visual colors and shapes while performing interaction behaviors like lifting, dropping, and looking on objects enables a robot to ground non-visual predicates like "empty" as well as visual predicates like "red". Previous work has established that grounding in multi-modal space improves performance on object retrieval from human descriptions. In this work, we gather behavior annotations from humans and demonstrate that these improve language grounding performance by allowing a system to focus on relevant behaviors for words like "white" or "halffull" that can be understood by looking or lifting, respectively. We also explore adding modality annotations (whether to focus on audio or haptics when performing a behavior), which improves performance, and sharing information between linguistically related predicates (if "green" is a color, "white" is a color), which improves grounding recall but at the cost of precision.

show abstract

“…Natural-language dialogue enables robots to ask clarification questions [30,59,84] and provide status updates [60,115] to conversational partners. Some of this research contributes to disambiguation methodologies [37,56,121], but a few attempt to renegotiate natural-language instructions that are not possible to execute [24,77,90].…”

Section: Dialogue Systems To Support Hrimentioning

confidence: 99%

Miscommunication Detection and Recovery in Situated Human–Robot Dialogue

Marge

Rudnicky

2019

ACM Trans. Interact. Intell. Syst.

View full text Add to dashboard Cite

Even without speech recognition errors, robots may face difficulties interpreting natural-language instructions. We present a method for robustly handling miscommunication between people and robots in task-oriented spoken dialogue. This capability is implemented in TeamTalk, a conversational interface to robots that supports detection and recovery from the situated grounding problems of referential ambiguity and impossible actions. We introduce a representation that detects these problems and a nearest-neighbor learning algorithm that selects recovery strategies for a virtual robot. When the robot encounters a grounding problem, it looks back on its interaction history to consider how it resolved similar situations. The learning method is trained initially on crowdsourced data but is then supplemented by interactions from a longitudinal user study in which six participants performed navigation tasks with the robot. We compare results collected using a general model to user-specific models and find that user-specific models perform best on measures of dialogue efficiency, while the general model yields the highest agreement with human judges. Our overall contribution is a novel approach to detecting and recovering from miscommunication in dialogue by including situated context, namely, information from a robot's path planner and surroundings. CCS Concepts: • Computing methodologies → Discourse, dialogue and pragmatics; • Humancentered computing → Natural language interfaces;

show abstract

Integrated Learning of Dialog Strategies and Semantic Parsing

Cited by 18 publications

References 19 publications

Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study

Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study

Guiding Interaction Behaviors for Multi-modal Grounded Language Learning

Miscommunication Detection and Recovery in Situated Human–Robot Dialogue

Contact Info

Product

Resources

About