SelFlow: Self-Supervised Learning of Optical Flow

Liu, Pengpeng; Lyu, Michael R.; King, Irwin; Xu, Jia

doi:10.1109/cvpr.2019.00470

Cited by 309 publications

(311 citation statements)

References 40 publications

Supporting

Mentioning

311

Contrasting

Order By: Relevance

“…SegFlow [40] trains an optical flow network and an image segmentation network jointly to improve the accuracy of both networks. DDFlow [41] and SelFlow [42] employ two data distillation methods and train the networks in an ''teacher-student'' manner. Although unsupervised algorithms have the advantage of not requiring labeled data, they often have complex training steps and lower accuracy than supervised methods.…”

Section: Cnn-based Optical Flowmentioning

confidence: 99%

Split-Attention Multiframe Alignment Network for Image Restoration

Liu

Feng

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Image registration (or image alignment), the problem of aligning multiple images with relative displacement, is a crucial step in many multiframe image restoration algorithms. To solve the problem that most existing image registration approaches can only align two images in one inference, we propose a splitattention multiframe alignment network (SAMANet). Pixel-level displacements between multiple images are first estimated at low-resolution scales and then refined gradually with the increase in feature resolution. To better integrate the interframe information, we present a split-attention module (SAM) and a dot-product attention module (DPAM), which can adaptively rescale the cost volume features and optical flow features according to the similarity between features from different images. The experimental results demonstrate the superiority of our SAMANet over state-of-the-art image registration methods in terms of both accuracy and robustness. To solve the ''ghosting effect'' caused by pixelwise registration, we designed two ''ghost'' removal modules: warping repetition detection module (WRDM) and attention fusion module (AFM). WRDM detects ''ghost'' regions during the image warping process without increasing the time complexity of the registration algorithm. AFM uses an attention mechanism to rescale the aligned images and enables the registration network and the subsequent image restoration networks to be trained jointly. To validate the strengths of the proposed approaches, we apply SAMANet, WRDM and AFM to three image/video restoration tasks. Extensive evaluations demonstrate that the proposed methods can enhance the performance of image restoration algorithms and outperform the other compared registration algorithms.

show abstract

Section: Cnn-based Optical Flowmentioning

confidence: 99%

Split-Attention Multiframe Alignment Network for Image Restoration

Liu

Feng

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The statistics for these two sets are shown in Fig. 7(d) and (e), where it can be observed that Table 2: The evaluation results on the KITTI flow benchmarks, where DDFlow [18], UnFlow [17], Flow2Stereo [45] and SelFlow [19] are the state-of-the-art self-supervised approaches. Best results are shown in bold font.…”

Section: Atg-pvd Datasetmentioning

confidence: 99%

“…Evaluation Since our ATG-PVD dataset does not contain optical flow ground truth, we evaluate our proposed SwiftFlow on the KITTI flow 2012 [46] and 2015 [44] benchmarks. According to the online leaderboard of the KITTI flow benchmarks, as shown in Table 2, our SwiftFlow ranks 24th on the KITTI flow 2012 3show the optical flow estimations and the corresponding error maps of (1) UnFlow [17], (2) SelFlow [19] and 3our SwiftFlow, respectively. Significantly improved regions are highlighted with green dashed boxes.…”

Section: Evaluation Of Swiftflowmentioning

confidence: 99%

See 1 more Smart Citation

ATG-PVD: Ticketing Parking Violations on A Drone

Wang¹,

Liu²,

Huang³

et al. 2020

Preprint

View full text Add to dashboard Cite

In this paper, we introduce a novel suspect-and-investigate framework, which can be easily embedded in a drone for automated parking violation detection (PVD). Our proposed framework consists of: 1) SwiftFlow, an efficient and accurate convolutional neural network (CNN) for unsupervised optical flow estimation; 2) Flow-RCNN, a flow-guided CNN for car detection and classification; and 3) an illegally parked car (IPC) candidate investigation module developed based on visual SLAM. The proposed framework was successfully embedded in a drone from ATG Robotics. The experimental results demonstrate that, firstly, our proposed SwiftFlow outperforms all other state-of-the-art unsupervised optical flow estimation approaches in terms of both speed and accuracy; secondly, IPC candidates can be effectively and efficiently detected by our proposed Flow-RCNN, with a better performance than our baseline network, Faster-RCNN; finally, the actual IPCs can be successfully verified by our investigation module after drone re-localization.

show abstract

“…However, their performance in challenging regions, such as partially occluded or texture-less regions, is often unsatisfactory [10,13]. The underlying cause of this performance degradation is threefold: 1) The popular coarse-to-fine framework [12,13] is often sensitive to noises in the flow initialization from the preceding pyramid level, and the challenging regions can introduce errors in the flow estimations, which in turn propagate to subsequent levels.…”

Section: Introductionmentioning

confidence: 99%

CoT-AMFlow: Adaptive Modulation Network with Co-Teaching Strategy for Unsupervised Optical Flow Estimation

Wang,

Fan,

Liu

2020

Preprint

View full text Add to dashboard Cite

The interpretation of ego motion and scene change is a fundamental task for mobile robots. Optical flow information can be employed to estimate motion in the surroundings. Recently, unsupervised optical flow estimation has become a research hotspot. However, unsupervised approaches are often easy to be unreliable on partially occluded or texture-less regions. To deal with this problem, we propose CoT-AMFlow in this paper, an unsupervised optical flow estimation approach. In terms of the network architecture, we develop an adaptive modulation network that employs two novel module types, flow modulation modules (FMMs) and cost volume modulation modules (CMMs), to remove outliers in challenging regions. As for the training paradigm, we adopt a co-teaching strategy, where two networks simultaneously teach each other about challenging regions to further improve accuracy. Experimental results on the MPI Sintel, KITTI Flow and Middlebury Flow benchmarks demonstrate that our CoT-AMFlow outperforms all other state-of-the-art unsupervised approaches, while still running in real time. Our project page is available at https://sites.google.com/view/cot-amflow.

show abstract

SelFlow: Self-Supervised Learning of Optical Flow

Cited by 309 publications

References 40 publications

Split-Attention Multiframe Alignment Network for Image Restoration

Split-Attention Multiframe Alignment Network for Image Restoration

ATG-PVD: Ticketing Parking Violations on A Drone

CoT-AMFlow: Adaptive Modulation Network with Co-Teaching Strategy for Unsupervised Optical Flow Estimation

Contact Info

Product

Resources

About