Attention-driven object detection and segmentation of cluttered table scenes using 2.5D symmetry

Abstract

The task of searching and grasping objects in cluttered scenes, typical of robotic applications in domestic environments requires fast object detection and segmentation. Attentional mechanisms provide a means to detect and prioritize processing of objects of interest. In this work, we combine a saliency operator based on symmetry with a segmentation method based on clustering locally planar surface patches, both operating on 2.5D point clouds (RGB-D images) as input data to yield a novel approach to table-top scene segmentation. Evaluation on indoor table-top scenes containing man-made objects clustered in piles and dumped in a box show that our approach to selection of attention points significantly improves performance of state-of-the-art attention-based segmentation methods.

Saliency-based object discovery on RGB-D data with a late-fusion approach

“…The difference to the here presented approach is that object candidates were purely generated based on color and not on depth data; depth was only used to create the 3D map. In the work of Potapova et al [23], the authors developed a method to segment objects from RGB-D images. A 3D symmetrybased saliency operator is used to select attention points.…”

“…In the robotics community, it is therefore preferred to generate a small set of object candidates. Many groups use the 3D information of the scene by either operating directly on the depth data from an RGB-D device [23] or by first reconstructing the scene and then doing the discovery of objects in the 3D reconstruction [11], [15]. Other approaches use information about changes over time to segregate objects from background [11] or interact with possible object candidates to determine what is an object [28].…”

Semantic segmentation priors for object discovery

“…Also Mishra and Aloimonos [12] use fixation points to locate the centre of objects and make use of the boundary-ownership concept in order to segment the objects. Potapova et al [13] rely on symmetry points in the depth map to find the centre of objects. However, as we showed in [3], methods such as the one of Potapova et al [13] have problems recalling objects in realistic cluttered scenes.…”

“…Potapova et al [13] rely on symmetry points in the depth map to find the centre of objects. However, as we showed in [3], methods such as the one of Potapova et al [13] have problems recalling objects in realistic cluttered scenes. Recently, Martín García et al [3] proposed a method that uses saliency as a cue to locate the presence of objects and segmentation of the scene in the colour and depth modalities independently to find object precise boundaries.…”

“…These are well-known descriptors of shape which are rotation and scale invariant. 8: A 3D convexity measure used in [3,13], computed as the average distance from the 3D points of the candidate to their closest face of the convex hull. 9: The normalized area of the object candidate mask.…”