CoT-AMFlow: Adaptive Modulation Network with Co-Teaching Strategy for Unsupervised Optical Flow Estimation

Abstract

The interpretation of ego motion and scene change is a fundamental task for mobile robots. Optical flow information can be employed to estimate motion in the surroundings. Recently, unsupervised optical flow estimation has become a research hotspot. However, unsupervised approaches are often easy to be unreliable on partially occluded or texture-less regions. To deal with this problem, we propose CoT-AMFlow in this paper, an unsupervised optical flow estimation approach. In terms of the network architecture, we develop an adaptive modulation network that employs two novel module types, flow modulation modules (FMMs) and cost volume modulation modules (CMMs), to remove outliers in challenging regions. As for the training paradigm, we adopt a co-teaching strategy, where two networks simultaneously teach each other about challenging regions to further improve accuracy. Experimental results on the MPI Sintel, KITTI Flow and Middlebury Flow benchmarks demonstrate that our CoT-AMFlow outperforms all other state-of-the-art unsupervised approaches, while still running in real time. Our project page is available at https://sites.google.com/view/cot-amflow.

Computer Stereo Vision for Autonomous Driving

“…In order to extrapolate the 3D information from a given driving scene, images from multiple views are required [27]. These images can be captured using either a single moving camera [28] or an array of synchronized cameras, as shown in Fig. 2.…”

“…2. The former is typically known as structure from motion (SfM) [29] or optical flow [28], while the latter is typically referred to as stereo vision or binocular vision (in case two cameras are used) [26]. SfM methods estimate both camera poses and the 3D points of interest from images captured from multiple views, which are linked by a collection of visual features.…”

“…Optical flow describes the motion of pixels between consecutive frames of a video sequence [31]. It is also regarded as an effective tool for dynamic object detection [28]. Stereo vision acquires depth information by finding the horizontal positional differences (disparities) of the visual feature correspondence pairs between two synchronously captured images.…”

SNE-RoadSeg+: Rethinking Depth-Normal Translation and Deep Supervision for Freespace Detection

“…Autonomous driving appears prominently in our society in the form of the advanced driver assistance system (ADAS) in both commercial and research vehicles [1]. Visual environment perception, the front-end module and key component of the ADAS, analyzes the raw data collected by the car's sensors and outputs its understanding to the driving scenario [2]- [4]. Its outputs are then used by other modules, such as prediction and planning, to ensure the safe navigation of self-driving cars in complex environments [5], [6].…”

Co-Teaching: an Ark to Unsupervised Stereo Matching

“…2, where two networks (LEAStereo [10] is used as the backbone network) with different initializations interactively teach each other about the occlusions. Our previous work has adopted this co-teaching framework for unsupervised optical flow estimation [17], and in this paper, we employ this framework for unsupervised stereo matching. This framework can significantly improve model's robustness against outliers and further enhance the overall performance of unsupervised stereo matching.…”