LanCon-Learn: Learning With Language to Enable Generalization in Multi-Task Manipulation

Silva, Andrew; Moorman, Nina; Silva, William; Zaidi, Zulfiqar; Gopalan, Nakul; Gombolay, Matthew

doi:10.1109/lra.2021.3139667

Cited by 12 publications

(7 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…"Go around the tree to your left and place the ball." Another popular alternative is language-conditioned learning, where language is employed to specify a reward function, or a task (Silva et al, 2021a;Andreas et al, 2017;Shridhar et al, 2022). Such approaches seek to improve the ability of an agent to complete a task(s) through intermediate language inputs, such as "take the ladder to your left".…”

Section: Learning Strategies From Languagementioning

confidence: 99%

“…In this paper, we develop an approach to solve a task we call automatic strategy translation , wherein we learn to infer strategic intent, in the form of goals and constraints, from language. Prior work has developed methods to utilize language to specify policies of an AI agent (Tambwekar et al, 2021;Gopalan et al, 2018;Thomason et al, 2019;Blukis et al, 2019) or specify reward functions or tasks which can be optimized for, via reinforcement learning (RL) or a planner (Gopalan et al, 2018;Padmakumar et al, 2021;Silva et al, 2021a). However, our work is the first to translate language into goals and constraints, which can be applied towards constrained optimization approaches for directing agent behavior independent of the original human specifier.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Computational Interface to Translate Strategic Intent from Unstructured Language in a Low-Data Setting

Tambwekar,

Dodeja,

Vaska

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Many real-world tasks involve a mixedinitiative setup, wherein humans and AI systems collaboratively perform a task. While significant work has been conducted towards enabling humans to specify, through language, exactly how an agent should complete a task (i.e., low-level specification), prior work lacks on interpreting the high-level strategic intent of the human commanders. Parsing strategic intent from language will allow autonomous systems to independently operate according to the user's plan without frequent guidance or instruction. In this paper, we build a computational interface capable of translating unstructured language strategies into actionable intent in the form of goals and constraints. Leveraging a game environment, we collect a dataset of over 1000 examples, mapping language strategies to the corresponding goals and constraints, and show that our model, trained on this dataset, significantly outperforms human interpreters in inferring strategic intent (i.e., goals and constraints) from language (p < 0.05). Furthermore, we show that our model (125M parameters) significantly outperforms ChatGPT for this task (p < 0.05) in a low-data setting.

show abstract

Section: Learning Strategies From Languagementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Computational Interface to Translate Strategic Intent from Unstructured Language in a Low-Data Setting

Tambwekar,

Dodeja,

Vaska

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

show abstract

“…[7] FAccT '22, June 21-24, 2022, Seoul, Republic of Korea Andrew Hundt, William Agnew, Vicky Zeng, Severin Kacianka, and Matthew Gombolay LfD warm-starts the process of synthesizing an "optimal" robot control policy with respect to a narrowly defined metric: The robot performs the easier, supervised learning task of imitating a human demonstrator followed by the more difficult problem of perfecting its behavior through RL [21]. Such approaches have been extended to 'zero-shot' settings where the robot is initially trained on a distribution of related tasks, then performs a novel task, such as through guidance from natural language instructions [98,100]. Many learning methods including zero-shot and transfer learning of robot skills continue to rapidly improve [19, 47-49, 93, 100, 111], often without loading dissolution models.…”

Section: Robotics and Ai With And Without Dissolution Modelsmentioning

confidence: 99%

Robots Enact Malignant Stereotypes

Hundt

Agnew

Zeng

et al. 2022

2022 ACM Conference on Fairness, Accountability, and Transparency

Self Cite

View full text Add to dashboard Cite

Stereotypes, bias, and discrimination have been extensively documented in Machine Learning (ML) methods such as Computer Vision (CV) [18,80], Natural Language Processing (NLP) [6], or both, in the case of large image and caption models such as OpenAI CLIP [14].In this paper, we evaluate how ML bias manifests in robots that physically and autonomously act within the world. We audit one of several recently published CLIP-powered robotic manipulation methods, presenting it with objects that have pictures of human faces on the surface which vary across race and gender, alongside task descriptions that contain terms associated with common stereotypes.Our experiments definitively show robots acting out toxic stereotypes with respect to gender, race, and scientifically-discredited physiognomy, at scale. Furthermore, the audited methods are less likely to recognize Women and People of Color. Our interdisciplinary sociotechnical analysis synthesizes across fields and applications such as Science Technology and Society (STS), Critical Studies, History, Safety, Robotics, and AI. We find that robots powered by large datasets and Dissolution Models (sometimes called "foundation models", e.g. CLIP) that contain humans risk physically amplifying malignant stereotypes in general; and that merely correcting disparities will be insufficient for the complexity and scale of the problem. Instead, we recommend that robot learning methods that physically manifest stereotypes or other harmful outcomes be paused, reworked, or even wound down when appropriate, until outcomes can be proven safe, effective, and just. Finally, we discuss comprehensive policy changes and the potential of new interdisciplinary research on topics like Identity Safety Assessment Frameworks and Design Justice to better understand and address these harms.

show abstract

“…Several approaches for LfD have been proposed, including Imitation Learning (IL) (Paleja et al 2020;Wang and Gombolay 2020;Silva et al 2021) and Inverse Reinforcement Learning (IRL) Gombolay 2020, 2021). Unlike IL's direct learning of a mapping from states to demonstrated actions, IRL infers the latent reward functions the demonstrators optimize for.…”

Section: Related Workmentioning

confidence: 99%

Strategy Discovery and Mixture in Lifelong Learning from Heterogeneous Demonstration

Jayanthi¹,

Chen²,

Gombolay³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Learning from Demonstration (LfD) approaches empower end-users to teach robots novel tasks via demonstrations of the desired behaviors, democratizing access to robotics. A key challenge in LfD research is that users tend to provide heterogeneous demonstrations for the same task due to various strategies and preferences. Therefore, it is essential to develop LfD algorithms that ensure flexibility (the robot adapts to personalized strategies), efficiency (the robot achieves sample-efficient adaptation), and scalability (robot reuses a concise set of strategies to represent a large amount of behaviors). In this paper, we propose a novel algorithm, Dynamic Multi-Strategy Reward Distillation (DMSRD), which distills common knowledge between heterogeneous demonstrations, leverages learned strategies to construct mixture policies, and continues to improve by learning from all available data. Our personalized, federated, and lifelong LfD architecture surpasses benchmarks in two continuous control problems with an average 77% improvement in policy returns and 42% improvement in log likelihood, alongside stronger task reward correlation and more precise strategy rewards.

show abstract

LanCon-Learn: Learning With Language to Enable Generalization in Multi-Task Manipulation

Cited by 12 publications

References 28 publications

A Computational Interface to Translate Strategic Intent from Unstructured Language in a Low-Data Setting

A Computational Interface to Translate Strategic Intent from Unstructured Language in a Low-Data Setting

Robots Enact Malignant Stereotypes

Strategy Discovery and Mixture in Lifelong Learning from Heterogeneous Demonstration

Contact Info

Product

Resources

About