Semantic and geometric reasoning for robotic grasping: a probabilistic logic approach

Abstract

While any grasp must satisfy the grasping stability criteria, good grasps depend on the specific manipulation scenario: the object, its properties and functionalities, as well as the task and grasp constraints. We propose a probabilistic logic approach for robot grasping, which improves grasping capabilities by leveraging semantic object parts. It provides the robot with semantic reasoning skills about the most likely object part to be grasped, given the task constraints and object properties, while also dealing with the uncertainty of visual perception and grasp planning. The probabilistic logic framework is task-dependent. It semantically reasons about pre-grasp configurations with respect to the intended task and employs object-task affordances and object/task ontologies to encode rules that generalize over similar object parts and object/task categories. The use of probabilistic logic for task-dependent grasping contrasts with current approaches that usually learn direct mappings from visual perceptions to taskdependent grasping points. The logic-based module receives data from a low-level module that extracts semantic objects parts, and sends information to the low-level grasp planner. These three modules define our probabilistic logic framework, which is able to perform robotic grasping in realistic kitchen-related scenarios. Response to Reviewers:The paper has been read carefully and small changes and corrections of the text have been made according to the review. We also added links where source code has been made available.

Robotic grasp detection using deep convolutional neural networks

“…Asif et al [23] introduce a consolidated framework known as EnsembleNet in which grasp generation network generates four grasp representations and Ensem-bleNet synthesizes these generated grasps to produce grasp scores from which the grasp with highest score gets selected. Antanas et al [24] discuss an interesting approach known as probabilistic logic framework that is said to improve the grasping capability of robot with the help of semantic object parts. This framework combines high-level reasoning with low-level grasping.…”

The State of Lifelong Learning in Service Robots:

“…An affordance for a certain grasp is generally synonymous with a high grasping quality, be it for a specific end-effector, angle or other purpose. In [7] and [78], objects' point clouds are semantically segmented by a rule-based system. Based on what the robot is tasked with, a specific segment of the object may be more suitable to be grasped than others.…”

“…On the opposite end of the spectrum, we may see the desire to get as much information from a single point of view as possible, in which case [156] provides a neural network based solution similar to Dex-Net for findign the ideal grasp with proper affordances. Instance-based learning with affordances [212] and applying affordances to segments of objects [7] are other possible options.…”

Feature Sensing and Robotic Grasping of Objects with Uncertain Information: A Review

“…As the typical movement is associated with a particular task, the previous sensor experiences could be used in the predictive model for subsequent task execution [ 89 ]. The grasping capability could be enhanced through leveraging the semantic object parts [ 90 ]. The pre-grasp configurations are reasoned with respect to the intended task.…”

“…Moreover, the tactile feedback provides tactile feedback for the robotic motion, while vision and other sensors are supplementary [ 130 , 131 ]. For unknown objects with occlusion, the point cloud data could be used to realize the active recognition of objects and then be applied for the semantic segmentation of objects [ 90 , 132 ]. The 3D deep CNN [ 133 ], as shown in Figure 15 , learns effective features from point clouds and classifies objects by classifier.…”