Semantic Image Segmentation via Deep Parsing Network

Abstract

This paper addresses semantic image segmentation by incorporating rich information into Markov Random Field (MRF), including high-order relations and mixture of label contexts. Unlike previous works that optimized MRFs using iterative algorithm, we solve MRF by proposing a Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which enables deterministic end-toend computation in a single forward pass. Specifically, DPN extends a contemporary CNN architecture to model unary terms and additional layers are carefully devised to approximate the mean field algorithm (MF) for pairwise terms. It has several appealing properties. First, different from the recent works that combined CNN and MRF, where many iterations of MF were required for each training image during back-propagation, DPN is able to achieve high performance by approximating one iteration of MF. Second, DPN represents various types of pairwise terms, making many existing works as its special cases. Third, DPN makes MF easier to be parallelized and speeded up in Graphical Processing Unit (GPU). DPN is thoroughly evaluated on the PASCAL VOC 2012 dataset, where a single DPN model yields a new state-of-the-art segmentation accuracy of 77.5%.

A deep learning model integrating FCNNs and CRFs for brain tumor segmentation

“…More recently, deep learning techniques have been adopted in brain tumor segmentation studies following their success in general image analysis fields, such as images classification (Krizhevsky et al, 2012), objects detection (Girshick et al, 2014), and semantic segmentation (Long et al, 2015 ; Zheng et al, 2015 ; Liu et al, 2015). Particularly, Convolutional Neural Networks (CNNs) were adopted for brain tumor image segmentation in the Multimodal Brain Tumor Image Segmentation Challenge (BRATS) 2014 (Zikic et al, 2014 ; Davy et al, 2014 ; Urban et al, 2014).…”

“…To take the local dependencies of labels into account, Havaei et al constructed a cascaded architecture by taking the pixel-wise probability segmentation results obtained by CNNs trained at early stages as additional input to their following CNNs (Havaei et al, 2015, 2017). To take into consideration appearance and spatial consistency of the segmentation results, Markov Random Fields (MRFs), particularly Conditional Random Fields (CRFs), have been integrated with deep learning techniques in image segmentation studies, either used as a post-process step of CNNs (Kamnitsas et al, 2017 ; Chen et al, 2014) or formulated as neural networks (Zheng et al, 2015 ; Liu et al, 2015). In the latter setting, both CNNs and MRFs/CRFs can be trained with back-propagation algorithms, tending to achieve better segmentation performance.…”

Deep Learning Markov Random Field for Semantic Segmentation

“…Our previous study [7] only shows the possibility on 2-Dimension image segmentation problem. Specifically, we employ dynamic node linking to construct graph in N -D space, which results in a model of N -D high-order MRF.…”

Recognition of Wheat Spike from Field Based Phenotype Platform Using Multi-Sensor Fusion and Improved Maximum Entropy Segmentation Algorithms

“…Image segmentation is a fundamental technique in image processing [42] and is the premise of object recognition and image interpretation. An image-segmentation problem typically involves extracting the consistent region and objects of interest from an image-processing process [43].…”