The recent introduction of consumer depth cameras has opened the way to novel segmentation approaches exploiting depth data together with the color information. This paper proposes a region merging segmentation scheme that jointly exploits the two clues. Firstly a set of multi-dimensional vectors is built considering the 3D spatial position, the surface orientation and the color data associated to each scene sample. Normalized cuts spectral clustering is applied to the obtained vectors in order to over-segment the scene into a large number of small segments. Then an iterative merging procedure is used to recombine the segments into the regions corresponding to the various objects and surfaces. The proposed algorithm tries to combine close compatible segments and uses a NURBS surface fitting scheme on the considered segments in order to understand if the regions candidate for the merging correspond to a single surface. The comparison with state-of-the-art methods shows how the proposed method provides an accurate and reliable scene segmentatio

Scene segmentation is a very challenging problem for which color information alone is often not sufficient. Recently the introduction of consumer depth cameras has opened the way to novel approaches exploiting depth data. This paper proposes a novel segmentation scheme that exploits the joint usage of color and depth data together with a 3D surface estimation scheme. Firstly a set of multi-dimensional vectors is built from color and geometry information and normalized cuts spectral clustering is applied to them in order to coarsely segment the scene. Then a NURBS model is fitted on each of the computed segments. The accuracy of the fitting is used as a measure of the plausibility that the segment represents a single surface or object. Segments that do not represent a single surface are split again into smaller regions and the process is iterated until the optimal segmentation is obtained. Experimental results show how the proposed method allows to obtain an accurate and reliable scene segmentation

No abstract

This paper proposes a joint color and depth segmentation scheme exploiting together geometrical clues and a learning stage. The approach starts from an initial over-segmentation based on spectral clustering. The input data is also fed to a Convolutional Neural Network (CNN) thus producing a per-pixel descriptor vector for each scene sample. An iterative merging procedure is then used to recombine the segments into the regions corresponding to the various objects and surfaces. The proposed algorithm starts by considering all the adjacent segments and computing a similarity metric according to the CNN features. The couples of segments with higher similarity are considered for merging. Finally the algorithm uses a NURBS surface fitting scheme on the segments in order to understand if the selected couples correspond to a single surface. The comparison with state-of-the-art methods shows how the proposed method provides an accurate and reliable scene segmentation

Among the various segmentation techniques, a widely used family of approaches are the ones based on region merging, where an initial oversegmentation is progressively refined by joining segments with similar characteristics. Instead of using deterministic approaches to decide which segments are going to be merged we propose to exploit a convolutional neural network which takes a couple of segments as input and decides whether to join or not the segments. We fitted this idea into an existent iterative semantic segmentation scheme for RGB-D data. We were able to lower the number of free parameters and to greatly speedup the procedure while achieving comparable or even higher results, thus allowing for its usage in free navigation systems. Furthermore, our method could be extended straightforwardly to other fields where region merging strategies are exploited. CCS CONCEPTS• Computing methodologies → Scene understanding; Image segmentation; Neural networks.