Planning with Learned Object Importance in Large Problem Instances using Graph Neural Networks

Silver, Tom; Chitnis, Rohan; Curtis, Aidan; Tenenbaum, Joshua B.; Lozano-Pérez, Tomás; Kaelbling, Leslie Pack

doi:10.1609/aaai.v35i13.17421

Cited by 28 publications

(8 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By contrast, efficient simulation means choosing the right idealization (Davis & Marcus, 2015;Fisher, 2006). In practice, systems that predict and plan over appropriately reduced representations are also more efficient (Agia et al, 2021;Silver et al, 2021). AI systems that reason more reasonably about physics could benefit from incorporating the same shortcuts that humans might be using, including limited samples (Battaglia et al, 2013;Hamrick et al, 2015), simplified shape representations (Smith et al, 2019;Ullman et al, 2017), or partial simulation as studied here.…”

Section: Discussionmentioning

confidence: 98%

Partial mental simulation explains fallacies in physical reasoning

Bass

Smith

Bonawitz

et al. 2021

Cognitive Neuropsychology

View full text Add to dashboard Cite

People can reason intuitively, efficiently, and accurately about everyday physical events.Recent accounts suggest that people use mental simulation to make such intuitive physical judgments. But mental simulation models are computationally expensive; how is physical reasoning relatively accurate, while maintaining computational tractability? We suggest that people make use of partial simulation, mentally moving forward in time only parts of the world deemed relevant. We propose a novel partial simulation model, and test it on the physical conjunction fallacy, a recently observed phenomenon (Ludwin-Peery, Bramley, Davis, & Gureckis, 2020) that poses a challenge for full simulation models. We find an excellent fit between our model's predictions and human performance on a set of scenarios that build on and extend those used by Ludwin-Peery et al. (2020), quantitatively and qualitatively accounting for deviations from optimal performance. Our results suggest more generally how we allocate cognitive resources to efficiently represent and simulate physical scenes.

show abstract

Section: Discussionmentioning

confidence: 98%

Partial mental simulation explains fallacies in physical reasoning

Bass

Smith

Bonawitz

et al. 2021

Cognitive Neuropsychology

View full text Add to dashboard Cite

show abstract

“…This allows such methods to create any feasible plan and not leverage commonsense knowledge to ensure that a goal state is reached in a few steps. Some recent works such as by Silver et al (2021) aim to learn object importance through human demonstrations. However, such methods can only work on objects seen previously in training and cannot generalize to unseen objects.…”

Section: Classical Planningmentioning

confidence: 99%

“…We ensure that in our domains the number of objects is large and they can be contained within/supported or transported by other objects as tools (details in Section 5). Similar to the work by Silver et al (2021), our model learns to prune-away irrelevant objects, additionally considering a domain with richer inter-object interactions. Consequently, our learner makes additional use of semantic properties and exploits correlations between actions and outputs interactions that are likely to lead to successful plans.…”

Section: Classical Planningmentioning

confidence: 99%

TOOLTANGO: Common sense Generalization in Predicting Sequential Tool Interactions for Robot Plan Synthesis

Tuli¹,

Rajas

Paul

et al. 2022

jair

View full text Add to dashboard Cite

Robots assisting us in environments such as factories or homes must learn to make use of objects as tools to perform tasks, for instance, using a tray to carry objects. We consider the problem of learning common sense knowledge of when a tool may be useful and how its use may be composed with other tools to accomplish a high-level task instructed by a human. Specifically, we introduce a novel neural model, termed TOOLTANGO, that first predicts the next tool to be used, and then uses this information to predict the next action. We show that this joint model can inform learning of a fine-grained policy enabling the robot to use a particular tool in sequence and adds a significant value in making the model more accurate. TOOLTANGO encodes the world state, comprising objects and symbolic relationships between them, using a graph neural network and is trained using demonstrations from human teachers instructing a virtual robot in a physics simulator. The model learns to attend over the scene using knowledge of the goal and the action history, finally decoding the symbolic action to execute. Crucially, we address generalization to unseen environments where some known tools are missing, but unseen alternative tools are present. We show that by augmenting the representation of the environment with pre-trained embeddings derived from a knowledge-base, the model can generalize effectively to novel environments. Experimental results show at least 48.8-58.1% absolute improvement over the baselines in predicting successful symbolic plans for a simulated mobile manipulator in novel environments with unseen objects. This work takes a step in the direction of enabling robots to rapidly synthesize robust plans for complex tasks, particularly in novel settings.

show abstract

“…At the end, the optimal plan uses 11 trucks. Even if 98.9% of the truck objects were irrelevant, they still impacted the performance of the planners (Fuentetaja and de la Rosa 2016; Silver et al 2021). Bob ends up frustrated with the whole procedure.…”

Section: Introductionmentioning

confidence: 99%

Planning with Object Creation

Corrêa,

De Giacomo,

Helmert

et al. 2024

ICAPS

View full text Add to dashboard Cite

Classical planning problems are defined using some specification language, such as PDDL. The domain expert defines action schemas, objects, the initial state, and the goal. One key aspect of PDDL is that the set of objects cannot be modified during plan execution. While this is fine in many domains, sometimes it makes modeling more complicated. This may impact the performance of planners, and it requires the domain expert to bound the number of required objects beforehand, which can be a challenge. We introduce an extension to the classical planning formalism, where action effects can create and remove objects. This problem is semi-decidable, but it becomes decidable if we can bound the number of objects in any given state, even though the state space is still infinite. On the practical side, we extend the Powerlifted planning system to support this PDDL extension. Our results show that this extension improves the performance of Powerlifted while supporting more natural PDDL models.

show abstract

Planning with Learned Object Importance in Large Problem Instances using Graph Neural Networks

Cited by 28 publications

References 28 publications

Partial mental simulation explains fallacies in physical reasoning

Partial mental simulation explains fallacies in physical reasoning

TOOLTANGO: Common sense Generalization in Predicting Sequential Tool Interactions for Robot Plan Synthesis

Planning with Object Creation

Contact Info

Product

Resources

About