In the Shadows, Shape Priors Shine: Using Occlusion to Improve Multi-region Segmentation

Abstract

We present a new algorithm for multi-region segmentation of 2D images with objects that may partially occlude each other. Our algorithm is based on the observation that human performance on this task is based both on prior knowledge about plausible shapes and taking into account the presence of occluding objects whose shape is already known -once an occluded region is identified, the shape prior can be used to guess the shape of the missing part. We capture the former aspect using a deep learning model of shape; for the latter, we simultaneously minimize the energy of all regions and consider only unoccluded pixels for data agreement.Existing algorithms incorporating object shape priors consider every object separately in turn and can't distinguish genuine deviation from the expected shape from parts missing due to occlusion. We show that our method significantly improves on the performance of a representative algorithm, as evaluated on both preprocessed natural and synthetic images. Furthermore, on the synthetic images, we recover the ground truth segmentation with good accuracy.

Learning Vector Quantized Shape Code for Amodal Blastomere Instance Segmentation

“…Some works exist on more natural images, e.g., Kihara et al [29] exploit occlusion as a signal to recover full masks for object instances via a Shape Boltzmann machine [30], but not for translucent objects. Li and Malik [16] introduce the first amodal segmentation method.…”

A Framework to Coordinate Segmentation and Recognition

“…Using a generative procedure to produce reference shape collaboratively may be more in line with human visual cognition mechanism. Several authors Kihara, Soloviev, and Chen 2016) proposed to use Restricted Boltzmann Machine to model multiple classes of shapes with global and local deformations and generate reference shapes. The similarity measurement between q and {q i } is avoided in these methods.…”