Robotic playing for hierarchical complex skill learning

Hangl, Simon; Uğur, Emre; Szedmák, Sándor; Piater, Justus

doi:10.1109/iros.2016.7759434

Cited by 24 publications

(49 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PS is a recent, physically-motivated RL model [26], which has already found several applications ranging from robotics [41] and quantum error correction [7] to the study of collective behavior [42] and automated experiment design [43]. Decision-making in PS occurs in a network of clips that constitutes the agent's episodic and compositional memory (ECM) (figure 2(a)).…”

Section: Projective Simulationmentioning

confidence: 99%

Photonic architecture for reinforcement learning

et al. 2020

View full text Add to dashboard Cite

The last decade has seen an unprecedented growth in artificial intelligence and photonic technologies, both of which drive the limits of modern-day computing devices. In line with these recent developments, this work brings together the state of the art of both fields within the framework of reinforcement learning. We present the blueprint for a photonic implementation of an active learning machine incorporating contemporary algorithms such as SARSA, Q-learning, and projective simulation. We numerically investigate its performance within typical reinforcement learning environments, showing that realistic levels of experimental noise can be tolerated or even be beneficial for the learning process. Remarkably, the architecture itself enables mechanisms of abstraction and generalization, two features which are often considered key ingredients for artificial intelligence. The proposed architecture, based on single-photon evolution on a mesh of tunable beamsplitters, is simple, scalable, and a first integration in quantum optical experiments appears to be within the reach of near-term technology.OPEN ACCESS RECEIVED promising features as compared to electronic processors [27][28][29][30]. For instance, nanosecond-scale routing and reconfigurability have already been demonstrated [31][32][33], while encoding information in photons enables decision-making at the speed of light, only limited by the generation and detection rates. Moreover, the use of phase-change materials for in-memory information processing [34] promises to enhance the energy efficiency, since their properties can be modified without continuous external intervention [35,36]. Importantly, since the architecture uses single photons, decision-making is fueled by genuine quantum randomness. This feature marks a fundamental departure from pseudorandom number generation in conventional devices. (ii) The second contribution is the development of a specific variant of PS based on binary decision trees (tree-PS, or t-PS for short), which is closely connected to the standard PS and suitable for the implementation on a photonic circuit. Furthermore, we discuss how this variant enables key features of artificial intelligence, namely abstraction and generalization [37,38].The Article is structured as follows. In section 2 we summarize the theoretical framework of RL, exemplified by three common approaches: SARSA, Q-learning, and PS. In section 3 we describe the blueprint for a fully integrated, photonic RL agent. We then numerically investigate its performance within two standard RL tasks and under realistic experimental imperfections in section 4. Finally, in section 5 we discuss promising features of this architecture within the context of t-PS.

show abstract

Section: Projective Simulationmentioning

confidence: 99%

Photonic architecture for reinforcement learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…C) Exploration beyond the modelled percept-action space can still be done, e.g. a book flipping action can be used to open a box [10]. Proposals for novel preparatory behaviours are considered proportional to their expected usefulness.…”

Section: Problem Statementmentioning

confidence: 99%

“…In previous work we introduced an approach that is loosely inspired by this paradigm [10]. The robot holds a set of sensing actions, preparatory behaviours and basic behaviours, i.e.…”

Section: Introductionmentioning

confidence: 99%

Skill Learning by Autonomous Robotic Playing Using Active Learning and Exploratory Behavior Composition

et al. 2020

Self Cite

View full text Add to dashboard Cite

We treat the problem of autonomous acquisition of manipulation skills where problem-solving strategies are initially available only for a narrow range of situations. We propose to extend the range of solvable situations by autonomous playing with the object. By applying previously-trained skills and behaviours, the robot learns how to prepare situations for which a successful strategy is already known. The information gathered during autonomous play is additionally used to learn an environment model. This model is exploited for active learning and the creative generation of novel preparatory behaviours. We apply our approach on a wide range of different manipulation tasks, e.g. book grasping, grasping of objects of different sizes by selecting different grasping strategies, placement on shelves, and tower disassembly. We show that the creative behaviour generation mechanism enables the robot to solve previously-unsolvable tasks, e.g. tower disassembly. We use success statistics gained during real-world experiments to simulate the convergence behaviour of our system. Experiments show that active improves the learning speed by around 9 percent in the book grasping scenario.

show abstract

“…Other well-known terminology is temporal projection [9], physics-based reasoning [10] and physical reasoning [11] which deal with predicting real-world behavior using knowledge inferred from physics simulation. In terms of repetitively performing actions which improve environment manipulation strategies, robotic playing [12] is another related bio-inspired technique which compares trial-and-error behavior while accumulating environment knowledge with a children's way of exploring the world. In the case of the proposed method, we can add the term dreaming robot to this list of synonyms for mental simulation because the manipulation strategy optimization happens during load-free times, for instance at night when the robot is otherwise unused.…”

Section: Related Workmentioning

confidence: 99%

Self-supervised damage-avoiding manipulation strategy optimization via mental simulation

Doernbach

2019

Intel Serv Robotics

View full text Add to dashboard Cite

Everyday robotics are challenged to deal with autonomous product handling in applications like logistics or retail, possibly causing damage on the items during manipulation. Traditionally, most approaches try to minimize physical interaction with goods. However, this paper proposes to take into account any unintended object motion and to learn damage-minimizing manipulation strategies in a self-supervised way. The presented approach consists of a simulation-based planning method for an optimal manipulation sequence with respect to possible damage. The planned manipulation sequences are generalized to new, unseen scenes in the same application scenario using machine learning. This learned manipulation strategy is continuously refined in a self-supervised, simulationin-the-loop optimization cycle during load-free times of the system, commonly known as mental simulation. In parallel, the generated manipulation strategies can be deployed in near-real time in an anytime fashion. The approach is validated on an industrial container-unloading scenario and on a retail shelf-replenishment scenario.

show abstract

Robotic playing for hierarchical complex skill learning

Cited by 24 publications

References 19 publications

Photonic architecture for reinforcement learning

Photonic architecture for reinforcement learning

Skill Learning by Autonomous Robotic Playing Using Active Learning and Exploratory Behavior Composition

Self-supervised damage-avoiding manipulation strategy optimization via mental simulation

Contact Info

Product

Resources

About