Robot Program Parameter Inference via Differentiable Shadow Program Inversion

Abstract

This paper presents Shadow Program Inversion with Differentiable Planning (SPI-DP), a novel first-order optimizer capable of optimizing robot programs with respect to both high-level task objectives and motion-level constraints.To that end, we introduce Differentiable Gaussian Process Motion Planning for N-DoF Manipulators (dGPMP2-ND), a differentiable collision-free motion planner for serial N-DoF kinematics, and integrate it into an iterative, gradient-based optimization approach for generic, parameterized robot program representations. SPI-DP allows first-order optimization of planned trajectories and program parameters with respect to objectives such as cycle time or smoothness subject to e.g. collision constraints, while enabling humans to understand, modify or even certify the optimized programs. We provide a comprehensive evaluation on two practical household and industrial applications.

Towards an Ontology for Robot Introspection and Metacognition

“…For example, in the robotic domain, particularly in bridging physical events to concepts thereof, it is imperative to recognise that interpretations take place: Any characterization of an objective occurrence unexceptionally depends on the observers' subjective narrative [49]. Such an interpretive view has been employed with great success in SOMA-flavoured NEEMs [10,11,12,13,14] and has also been argued to be propitious for classifying mental processes [50]. SOMA enforces this stance by building upon the foundational ontology DUL, which consistently distinguishes PhysicalEntities from SocialEntities that exist "for the sake of [.…”

“…For different learning tasks, selected parts can then later be queried via the free Open-EASE platform [10]. This has proved useful, e.g., for learning action parameterization [11,12], learning common-sense knowledge from humans in VR [13], and transferring experiences between robots and affordances to novel objects [14].…”

Bayesian optimization with unknown constraints in graphical skill models for compliant manipulation tasks using an industrial robot

“…Mitsioni et al (2021) instead proposed to learn the environment dynamics from an NN in order to apply a model predictive control, if the current state is classified as safe via a GP classifier. Alt et al (2021) also applied NNs via differentiable shadow programs that employ the parameterization of robotic skills in the form of Cartesian poses and wrenches in order to achieve force-sensitive manipulation skills, even on industrial robots. They include the success probability in the output of the NNs, in order to minimize the failure rate.…”

LapSeg3D: Weakly Supervised Semantic Segmentation of Point Clouds Representing Laparoscopic Scenes

“…CRAM [5] proposes a knowledge-based approach to robot programming by combining a symbolic program and knowledge representation with the KnowRob knowledge representation and reasoning (KR&R) engine [6] and a realistic simulation environment. This paradigm has been shown to support several forms of AI-assisted robot programming [7], or synthesis of robot programs from human virtual reality (VR) demonstrations [8]. Several recent approaches propose to leverage natural language as a suitable abstraction for program synthesis.…”