Deep Online Video Stabilization With Multi-Grid Warping Transformation Learning

Wang, Miao; Yang, Guo-Ye; Lin, Jia-Rui; Zhang, Song-Hai; Shamir, Ariel; Lu, Shao-Ping; Hu, Shi‐Min

doi:10.1109/tip.2018.2884280

Cited by 109 publications

(97 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the popularity of deep learning, there are also some video stabilization methods based on deep learning. Input stabilized and jitter video to the network, and output a homography matrix to the network [ 21 ]. The objective function does not consider the effects of multi-object and parallax, so it is only effective for a single object or background shaky video.…”

Section: Related Workmentioning

confidence: 99%

Robust Global Motion Estimation for Video Stabilization Based on Improved K-Means Clustering and Superpixel

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

Obtaining accurate global motion is a crucial step for video stabilization. This paper proposes a robust and simple method to implement global motion estimation. We don’t extend the framework of 2D video stabilization but add a “plug and play” module to motion estimation based on feature points. Firstly, simple linear iterative clustering (SLIC) pre-segmentation is used to obtain superpixels of the video frame, clustering is performed according to the superpixel centroid motion vector and cluster center with large value is eliminated. Secondly, in order to obtain accurate global motion estimation, an improved K-means clustering is proposed. We match the feature points of the remaining superpixels between two adjacent frames, establish a feature points’ motion vector space, and use improved K-means clustering for clustering. Finally, the richest cluster is being retained, and the global motion is obtained by homography transformation. Our proposed method has been verified on different types of videos and has efficient performance than traditional approaches. The stabilization video has an average improvement of 0.24 in the structural similarity index than the original video and 0.1 higher than the traditional method.

show abstract

Section: Related Workmentioning

confidence: 99%

Robust Global Motion Estimation for Video Stabilization Based on Improved K-Means Clustering and Superpixel

Zhang

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…One seeks to directly estimate the camera path (position), and the video stabilization can be considered as a camera path smoothing problem [36]. This formulation aims to stabilize homographic distortion cause by camera shaking, and recently a deep learning based method has been developed to learn from data registered by a mechanical stabilizer [37], which shows greater efficiency than traditional algorithms. The other type of approach models the instability of the video (or frame stream) as an appearance change [38].…”

Section: Video Stabilizationmentioning

confidence: 99%

Data Stream Stabilization for Optical Coherence Tomography Volumetric Scanning

Liao

Mora

Rosa

et al. 2021

IEEE Trans. Med. Robot. Bionics

View full text Add to dashboard Cite

Optical Coherence Tomography (OCT) is an emerging medical imaging modality for luminal organ diagnosis. The non-constant rotation speed of optical components in the OCT catheter tip causes rotational distortion in OCT volumetric scanning. By improving the scanning process, this instability can be partially reduced. To further correct the rotational distortion in the OCT image, a volumetric data stabilization algorithm is proposed. The algorithm first estimates the Non-Uniform Rotational Distortion (NURD) for each B-scan by using a Convolutional Neural Network (CNN). A correlation map between two successive B-scans is computed and provided as input to the CNN. To solve the problem of accumulative error in iterative frame stream processing, we deploy an overall rotation estimation between reference orientation and actual OCT image orientation. We train the network with synthetic OCT videos by intentionally adding rotational distortion into real OCT images. As part of this article we discuss the proposed method in two different scanning modes: the first is a conventional pullback mode where the optical components move along the protection sheath, and the second is a self-designed scanning mode where the catheter is globally translated by using an external actuator. The efficiency and robustness of the proposed method are evaluated with synthetic scans as well as real scans under two scanning modes.

show abstract

“…More recently, convolutional neural networks have been applied to both registration [18] and stabilization [16] domains. Both methods have proposed to use a rich warping model based on Thin Plate Splines (TPS).…”

Section: Prior Workmentioning

confidence: 99%

“…1.a). Authors of StabNet [16] based their approach on a siamese convolutional network that was trained thanks to a stabilization database. This database was acquired with the help of a single handling device to which 2 different cameras were attached, only one of which was physically stabilized with a gimbal.…”

Section: Prior Workmentioning

confidence: 99%

See 1 more Smart Citation

Jitter-Free Registration for Unmanned Aerial Vehicle Videos

Lemaire¹,

Crispim-Junior²,

Robinault³

et al. 2019

Advances in Visual Computing

View full text Add to dashboard Cite

Unmanned Aerial Vehicles (UAVs), such as tethered drones, become increasingly popular for video acquisition, within video surveillance or remote, scientific measurement contexts. However, UAV recordings often present an unstable, variable viewpoint that is detrimental to the automatic exploitation of their content. This is often countered by one amongst two strategies, video registration and video stabilization, which are usually affected by distinct issues, namely jitter and drifting. This paper proposes a hybrid solution between both techniques that produces a jitter-free registration. A lightweight implementation enables real time, automatic generation of videos with a constant viewpoint from unstable video sequences acquired with stationary UAVs. Performance evaluation is carried out using video recordings from traffic surveillance scenes up to 15 minutes long, including multiple mobile objects.

show abstract

Deep Online Video Stabilization With Multi-Grid Warping Transformation Learning

Cited by 109 publications

References 21 publications

Robust Global Motion Estimation for Video Stabilization Based on Improved K-Means Clustering and Superpixel

Robust Global Motion Estimation for Video Stabilization Based on Improved K-Means Clustering and Superpixel

Data Stream Stabilization for Optical Coherence Tomography Volumetric Scanning

Jitter-Free Registration for Unmanned Aerial Vehicle Videos

Contact Info

Product

Resources

About