Vision-based grasp learning of an anthropomorphic hand-arm system in a synergy-based control framework

Abstract

In this work, the problem of grasping novel objects with an anthropomorphic hand-arm robotic system is considered. In particular, an algorithm for learning stable grasps of unknown objects has been developed based on an object shape classification and on the extraction of some associated geometric features. Different concepts, coming from fields such as machine learning, computer vision, and robot control, have been integrated together in a modular framework to achieve a flexible solution suitable for different applications. The results presented in this work confirm that the combination of learning from demonstration and reinforcement learning can be an interesting solution for complex tasks, such as grasping with anthropomorphic hands. The imitation learning provides the robot with a good base to start the learning process that improves its abilities through trial and error. The learning process occurs in a reduced dimension subspace learned upstream from human observation during typical grasping tasks. Furthermore, the integration of a synergy-based control module allows reducing the number of trials owing to the synergistic approach.

Leveraging Human Perception in Robot Grasping and Manipulation Through Crowdsourcing and Gamification

“…Human perception is for instance unparalleled in the estimation of object affordances or “action possibilities” (Gibson, 1979 ; Montesano et al, 2008 ; Sun et al, 2010 ). Affordances play a major role in manipulation planning as they determine the appropriate grasp types for particular objects, which can be very beneficial for frameworks relying on vision (Zeng et al, 2018 ; Ficuciello et al, 2019 ). Visual estimation of object characteristics other than the class was presented through the exoskeleton glove experiments, where the chosen object attribute was its stiffness.…”

A Comprehensive Approach to the Generation of Human-Like Arm Movements on Robot NAO

“…The complex arm movements can be decomposed into different simple motion models through MPs, which makes great facilitation to generate human-like movements of anthropomorphic arms. According to (2), there are 10 MPs as shown in Table 1. Theses MPs can be divided into two types: Motion Movement Primitive (MMP) and Function Movement Primitive (FMP).…”

“…After solving the IK problem, the robot can perform the task based on the selected MP. (10) In fact, as shown in Section Ⅱ, every MP has a corresponding Primitive Matrix e. When the maximum g is selected, the corresponding Primitive Matrix can be obtained through (2). In order to visualize the result of motiondecision, the Primitive Matrix can be depicted graphically.…”

“…With the development of the robot technology and the expanding of the application area, new generations of humanoid robots are designed to interact and communicate with humans in a natural and friendly way [1]. The anthropoid shape gives humanoid robots the advantages over other robots in HRI and makes them more accessible for humans [2]. In HRI, humanoid robots work with humans and even can be the companion of humans.…”

Continuous learning of emergent behavior in robotic matter

“…Toward this goal, reinforced learning strategies can be used to deal with more complex tasks that cannot directly be programmed in the behavior of the robot. By sensing their environment, robots can build models of themselves ( 4 ) or optimize human-made models with machine learning strategies to improve their behavior ( 1 , 2 , 4 – 8 ). Apart from a clear division between training and task execution, these systems rely on a centralized architecture that—with an increasing number of active components—demands advanced models and higher computational power.…”