Multi-Class Multi-Scale Series Contextual Model for Image Segmentation

Abstract

Contextual information has been widely used as a rich source of information to segment multiple objects in an image. A contextual model uses the relationships between the objects in a scene to facilitate object detection and segmentation. Using contextual information from different objects in an effective way for object segmentation, however, remains a difficult problem. In this paper, we introduce a novel framework, called multiclass multiscale (MCMS) series contextual model, which uses contextual information from multiple objects and at different scales for learning discriminative models in a supervised setting. The MCMS model incorporates cross-object and inter-object information into one probabilistic framework and thus is able to capture geometrical relationships and dependencies among multiple objects in addition to local information from each single object present in an image. We demonstrate that our MCMS model improves object segmentation performance in electron microscopy images and provides a coherent segmentation of multiple objects. Through speeding up the segmentation process, the proposed method will allow neurobiologists to move beyond individual specimens and analyze populations paving the way for understanding neurodegenerative diseases at the microscopic level.

A modular hierarchical approach to 3D electron microscopy image segmentation

“…A membrane detection map ℬ = { b i } is a probability image with each pixel intensity b i ∈ [0, 1] as shown in Figure 1(b). Such membrane detection maps are commonly obtained using machine learning algorithms, such as (Jain et al, 2007; Laptev et al, 2012; Ciresan et al, 2012; Seyedhosseini et al, 2013). We can obtain a segmentation of the cells in the image by simply thresholding the probability map.…”

“…The boundary classifier takes 88 non-local features computed from each pair of merging regions, including region geometry, image intensity, and texture statistics from both original EM images and membrane detection probability maps, and merging saliency information (see Appendix B for a summary of features). One advantage of our features over the local features commonly used by pixel classifiers (Laptev et al, 2012; Seyedhosseini et al, 2013) is that our features are extracted from regions instead of pixels and thus can be more informative. For instance, we use geometric features to incorporate region shape information for the classification procedure, which is not feasible for pixel classifiers.…”

Semantic Image Segmentation with Contextual Hierarchical Models

“…Second row: Horse segmentation (Weiz-mann dataset [17]. Third row: Membrane detection (mouse neuropil dataset [9]). Fourth row: Edge detection (Berkeley dataset [19]).…”

“…However, they have a drawback that they do not obtain contextual information at multiple scales. Multi-scale processing of images has been proven critical in many computer vision tasks [8], [9]. OWT-UCM [10] takes advantage of processing the input image at multiple scales through a hierarchy.…”

“…In all cases, CHM results in either state-of-the-art or near state-of-the-art performance. In addition, we apply CHM on two electron microscopy datasets for cell membrane detection (Drosophila VNC [21], [22] and mouse neuropil [9]). CHM outperforms many existing algorithms for membrane detection and can be used as the first step towards reconstruction of the connectome, i.e., the map of neural connectivity in the mammalian nervous system [23].…”