A Perceptual Depth Shape-based CRF Model for Deformable Surface Labeling

Abstract

Real-time deformable scene understanding is a challenging task. In this paper, we address this problem by using Conditional random fields (CRFs) framework and perceptual shape salience occupancy patterns. CRF is a powerful probabilistic model that has been widely used for labeling image segments. It is particularly well-suited to modeling local interactions and global consistency among bottom-up regions (e.g. superpixels). However, its capacity could be limited if the underlying feature potentials are not well reflecting the scene properties. We propose a depth shapebased CRF model for deformable surface (sand in our case) labeling by utilizing expressive novel shape salience occupancy patterns (SOP).Experimental results demonstrate the effectiveness and robustness of the method on recorded video datasets. While our work has concentrated on sand surface labeling, the approach can be applied to other surface materials (e.g. snow, mud), and extended to non-planar surfaces as well (e.g. sculpting blocks).

Doing While Thinking

“…4 major technical steps are applied to support this: superpixel segmentation, color, shape and texture feature extraction, conditional random field (CRF)-based probabilistic framework for classification (see Figure 3). The technical details of this sand labelling method can be found in [36].…”