Ontology-Assisted Generalisation of Robot Action Execution Knowledge

Mitrevski, Alex; Plöger, Paul G.; Lakemeyer, Gerhard

doi:10.1109/iros51168.2021.9636791

Cited by 9 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Why did the human want the juice and what is a fitting alternative?) • Location Detection (Welke et al, 2013): Categorize the location based on the recognized objects (e.g., the robot detects milk and juice and concludes that the location is a fridge) • Navigation (Shylaja et al, 2013;Li et al, 2022): Navigate to a specific location • Object Delivery (Lam et al, 2012;Riazuelo et al, 2013;Mühlbacher and Steinbauer, 2014;Al-Moadhen et al, 2015;Zhang and Stone, 2015;Wang et al, 2019;Yang et al, 2019): Finding the requested object and delivering it to a specific location • Object Localization (Varadarajan and Vincze, 2012b;Zhou et al, 2012;Kaiser et al, 2014;Riazuelo et al, 2015;Jebbara et al, 2018;Daruna et al, 2019;Zhang et al, 2019;Chernova et al, 2020): Finding a specific object in an (unknown) environment • Object Recognition (Daoutis et al, 2012;Pratama et al, 2014;Kümpel et al, 2020;Chiatti et al, 2022): Recognize a specific object based on its properties • Pick and Place (Al-Moadhen et al, 2013;Javia and Cimiano, 2016;Mitrevski et al, 2021): Pick an object up and place it at a different location • Reminiscence Therapy (Wu et al, 2019): Asking questions about provided pictures to get the human to remember and socialize • Table Setting (Salinas Pinacho et al, 2018;Haidu and Beetz, 2019): Set the table for a meal scenario (and maybe also clean up afterwards) • Tidy Up (Aker et al, 2012;Skulkittiyut et al, 2013): Bring a specified part of the environment in order by removing unusual objects • Tool Substitution (Zhu et al, 2015;Thosar et al, 2020;2021;…”

Section: Use Cases and Their Application Domainmentioning

confidence: 99%

Commonsense knowledge in cognitive robotics: a systematic literature review

Töberg,

Ngonga Ngomo,

Beetz

et al. 2024

Front. Robot. AI

View full text Add to dashboard Cite

One of the big challenges in robotics is the generalization necessary for performing unknown tasks in unknown environments on unknown objects. For us humans, this challenge is simplified by the commonsense knowledge we can access. For cognitive robotics, representing and acquiring commonsense knowledge is a relevant problem, so we perform a systematic literature review to investigate the current state of commonsense knowledge exploitation in cognitive robotics. For this review, we combine a keyword search on six search engines with a snowballing search on six related reviews, resulting in 2,048 distinct publications. After applying pre-defined inclusion and exclusion criteria, we analyse the remaining 52 publications. Our focus lies on the use cases and domains for which commonsense knowledge is employed, the commonsense aspects that are considered, the datasets/resources used as sources for commonsense knowledge and the methods for evaluating these approaches. Additionally, we discovered a divide in terminology between research from the knowledge representation and reasoning and the cognitive robotics community. This divide is investigated by looking at the extensive review performed by Zech et al. (The International Journal of Robotics Research, 2019, 38, 518–562), with whom we have no overlapping publications despite the similar goals.

show abstract

Section: Use Cases and Their Application Domainmentioning

confidence: 99%

Commonsense knowledge in cognitive robotics: a systematic literature review

Töberg,

Ngonga Ngomo,

Beetz

et al. 2024

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…FOON [8] is a structured method for representing the knowledge the models' objects and their movements in manipulation tasks, constructed through the manual annotation of instructional videos. Instead of a comprehensive symbolic representation system, several works apply knowledge to different robotics tasks, such as vision [9], grasping [10,11], assembly [12], and path planning [13,14], to describe robots' behavior processes. Furthermore, the utilization of knowledge graph embedding enables inference and finds application in the field of robotics [15].…”

Section: Knowledge Representation In Robotic Manipulationmentioning

confidence: 99%

Hierarchical Understanding in Robotic Manipulation: A Knowledge-Based Framework

Miao,

Jia,

Sun

et al. 2024

Actuators

View full text Add to dashboard Cite

In the quest for intelligent robots, it is essential to enable them to understand tasks beyond mere manipulation. Achieving this requires a robust parsing mode that can be used to understand human cognition and semantics. However, the existing methods for task and motion planning lack generalization and interpretability, while robotic knowledge bases primarily focus on static manipulation objects, neglecting the dynamic tasks and skills. To address these limitations, we present a knowledge-based framework for hierarchically understanding various factors and knowledge types in robotic manipulation. Using this framework as a foundation, we collect a knowledge graph dataset describing manipulation tasks from text datasets and an external knowledge base with the assistance of large language models and construct the knowledge base. The reasoning tasks of entity alignment and link prediction are accomplished using a graph embedding method. A robot in real-world environments can infer new task execution plans based on experience and knowledge, thereby achieving manipulation skill transfer.

show abstract

“…To generalize the obtained models, they consider an ontology to find how closely related these objects are, but they don't consider object variables like the material. A related approach [15] defines object clusters with similar action success probability via ontologies. However, they also note that other information like object affordances or object materials could be used for generalization.…”

Section: B Learning Explainable Models Of Cause-effect Relationsmentioning

confidence: 99%

Why did I fail? A Causal-based Method to Find Explanations for Robot Failures

Diehl¹,

Ramirez-Amaro²

2022

Preprint

View full text Add to dashboard Cite

Robot failures in human-centered environments are inevitable. Therefore, the ability of robots to explain such failures is paramount for interacting with humans to increase trust and transparency. To achieve this skill, the main challenges addressed in this paper are I) acquiring enough data to learn a cause-effect model of the environment and II) generating causal explanations based on that model. We address I) by learning a causal Bayesian network from simulation data. Concerning II), we propose a novel method that enables robots to generate contrastive explanations upon task failures. The explanation is based on setting the failure state in contrast with the closest state that would have allowed for successful execution, which is found through breadth-first search and is based on success predictions from the learned causal model. We assess the sim2real transferability of the causal model on a cube stacking scenario. Based on real-world experiments with two differently embodied robots, we achieve a sim2real accuracy of 70% without any adaptation or retraining. Our method thus allowed real robots to give failure explanations like, 'the upper cube was dropped too high and too far to the right of the lower cube.'

show abstract

Ontology-Assisted Generalisation of Robot Action Execution Knowledge

Cited by 9 publications

References 24 publications

Commonsense knowledge in cognitive robotics: a systematic literature review

Commonsense knowledge in cognitive robotics: a systematic literature review

Hierarchical Understanding in Robotic Manipulation: A Knowledge-Based Framework

Why did I fail? A Causal-based Method to Find Explanations for Robot Failures

Contact Info

Product

Resources

About