Online Self-Supervised Learning for Object Picking: Detecting Optimum Grasping Position using a Metric Learning Approach

Abstract

Online self-supervised learning methods are attractive candidates for automatic object picking. Self-supervised learning collects training data online during the learning process. However, the trial samples lack the complete ground truth because the observable parts of the agent are limited. That is, the information contained in the trial samples is often insufficient to learn the specific grasping position of each object. Consequently, the training falls into a local solution, and the grasp positions learned by the robot are independent of the state of the object. In this study, the optimal grasping position of an individual object is determined from the grasping score, defined as the distance in the feature space obtained using metric learning. The closeness of the solution to the pre-designed optimal grasping position was evaluated in trials. The proposed method incorporates two types of feedback control: one feedback enlarges the grasping score when the grasping position approaches the optimum; the other reduces the negative feedback of the potential grasping positions among the grasping candidates. The proposed online self-supervised learning method employs two deep neural networks. : a single shot multibox detector (SSD) that detects the grasping position of an object, and Siamese networks (SNs) that evaluate the trial sample using the similarity of two input data in the feature space. Our method embeds the relation of each grasping position as feature vectors by training the trial samples and a few pre-samples indicating the optimum grasping position. By incorporating the grasping score based on the feature space of SNs into the SSD training process, the method preferentially trains the optimum grasping position. In the experiment, the proposed method achieved a higher success rate than the baseline method using simple teaching signals. And the grasping scores in the feature space of the SNs accurately represented the grasping positions of the objects.

Multi-Target Coordinated Search Algorithm for Swarm Robotics Considering Practical Constraints

“…In this paper, the target search problem of swarm robots in unknown complex environments is mainly studied, such as forest fire detection (Yao et al, 2018 ; Marzaeva, 2019 ), toxic gas leak detection (Zhang et al, 2010 ; Moshayedi and Gharpure, 2013 ), search and rescue of missing personnel (Goodrich et al, 2009 ; Kamegawa et al, 2020 ), military target detection (Ha and Cho, 2018 ; Jiong et al, 2019 ) and so on. In order to solve this type of search problem, there are mainly composed of two main categories of design strategies, namely, behavior-based search and learning-based search (Cizek and Faigl, 2019 ; Berscheid et al, 2020 ; Suzuki et al, 2020 ), and this article mainly discusses the former.…”

Stable deep reinforcement learning method by predicting uncertainty in rewards as a subtask

“…Furthermore, we can use extracted features as the target signals of another network. For example, some studies use the feature values of images of target values as a signal to optimize robot behaviors [8,21]. When employing a reward signal to acquire advanced behavior, a variety of noise types not intended by the experimenter will occur.…”