Automatic layered RGB‐D scene flow estimation with optical flow field constraint

Abstract

Scene flow estimation with RGB-D frames is receiving increasing attention in digital video processing and computer vision due to the widespread use of depth sensors. Existing methods based on object segmentation have shown their effectiveness for object occlusion and large displacement. However, improper segmentation often causes incomplete segmented areas or incorrect edges, which will result in inaccurate occlusion inference and scene flow estimation. To this end, an automatic layered RGB-D scene flow estimation method is proposed, which achieves more accurate layering of objects in depth image by exploring motion information. The authors exploit super-pixel segmentation for initial layering, which is beneficial for preserving edges and integrity of the objects. Furthermore, an optical flow which is highly correlated with the scene is also used for automatic layering. Finally, the utilisation of super-pixel segmentation and motion information would ensure the integrity of the object area and improve the accuracy of scene flow. They have validated their approach both qualitatively and quantitatively on several public datasets. Experimental results show that the proposed method is able to preserve the integrity of the object and achieve lower root mean square error and average angular error as compared with the current state-of-the-art algorithms. 10 IET Image Process.

FAFNet: Fully aligned fusion network for RGBD semantic segmentation based on hierarchical semantic flows

“…Optical flow is a widely-used image registration method, it is originally proposed in motion detection to describe the motion of an observed target, surface, or edge caused by motion relative to the observer. In recent years, optical flow has been used more and more extensively in computer vision-related tasks [27]. Semantic flow proposed by Li et al extends the concept of optical flow to semantic segmentation [4].…”