Local Occlusion Detection under Deformations Using Topological Invariants

Abstract

Abstract. Occlusions provide critical cues about the 3D structure of man-made and natural scenes. We present a mathematical framework and algorithm to detect and localize occlusions in image sequences of scenes that include deforming objects. Our occlusion detector works under far weaker assumptions than other detectors. We prove that occlusions in deforming scenes occur when certain well-defined local topological invariants are not preserved. Our framework employs these invariants to detect occlusions with a zero false positive rate under assumptions of bounded deformations and color variation. The novelty and strength of this methodology is that it does not rely on spatio-temporal derivatives or matching, which can be problematic in scenes including deforming objects, but is instead based on a mathematical representation of the underlying cause of occlusions in a deforming 3D scene. We demonstrate the effectiveness of the occlusion detector using image sequences of natural scenes, including deforming cloth and hand motions.

Learning to find occlusion regions

“…They iteratively improve their occlusion map by adjusting a color segmentation map. Lobaton et al [25] also employed color information in finding occlusions. They introduced an image homeomorphism criterion for detecting local occlusions when objects and backgrounds can clearly be segmented into connected components based on distinct color distributions.…”

A survey of variational and CNN-based optical flow techniques

“…A promising avenue for improvement is in the combination of optical flow features such as motion constancy [127] with image features (e.g. brightness, color, texture, depth) [81,124,[127][128][129].…”

Robust topological features for deformation invariant image matching

“…Our method on the other hand is provably robust to the presence of noise since it operates over several inter-level sets of the intensity image simultaneously. This paper is motivated by our recent work that defines a topological description of occlusions during deformation [7].…”