Recurrent Pixel Embedding for Instance Grouping

Abstract

We introduce a differentiable, end-to-end trainable framework for solving pixel-level grouping problems such as instance segmentation consisting of two novel components. First, we regress pixels into a hyper-spherical embedding space so that pixels from the same group have high cosine similarity while those from different groups have similarity below a specified margin. We analyze the choice of embedding dimension and margin, relating them to theoretical results on the problem of distributing points uniformly on the sphere. Second, to group instances, we utilize a variant of mean-shift clustering, implemented as a recurrent neural network parameterized by kernel bandwidth. This recurrent grouping module is differentiable, enjoys convergent dynamics and probabilistic interpretability. Backpropagating the group-weighted loss through this module allows learning to focus on only correcting embedding errors that won't be resolved during subsequent clustering. Our framework, while conceptually simple and theoretically abundant, is also practically effective and computationally efficient. We demonstrate substantial improvements over stateof-the-art instance segmentation for object proposal generation, as well as demonstrating the benefits of grouping loss for classification tasks such as boundary detection and semantic segmentation.

Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth

“…To ensure that during samplingσ k ≈ σ k = 1 |S k | σi∈S k σ i , we add a smoothness term for each instance to the total loss: 12)…”

Rethinking Task and Metrics of Instance Segmentation on 3D Point Clouds

“…Approaches labeling an object identifier directly to each pixel are called IC, and some studies have been conducted in this area for image segmentation [5,12,14]. Brabandere et al [5] proposed a discriminative loss function that learns a mapping to an embedding space where the embeddings form clusters for each object.…”

“…Such a feature learning method that trains the embedding to minimize the distance between embeddings with the same semantics while maximizing the distance between embeddings with different semantics is widely used in category classification [3,23] and similarity learning [11,20]. This concept has been used for recent instance segmentation studies on images such as those on discriminative loss [5], [12]. Inspired by this, we propose a novel instance segmentation method that overcomes the discriminative loss problem.…”

“…In addition, L dist attempts to increase distance between cluster centroids when the cosine similarity is larger than δ d . In an image recognition study ( [12]), the feature was also learned using the cosine similarity using a unit hypersphere. However, in that study, similarities between all pairs of points were calculated, whereas our method only considers the similarities between points and the corresponding cluster centroids.…”

Learning Object-Specific Distance From a Monocular Image

“…With the advances in the field of computer vision, visual environment perception, which includes object classification, detection, segmentation and distance estimation, has become a key component in the development of autonomous driving cars. Although researchers have paid a lot of efforts on improving the accuracy of visual perception, they mainly focus on more popular tasks, such as object classification, detection and segmentation [29,27,17]. * indicates corresponding author.…”