Improving Video Stabilization Using Multi-Resolution MSER Features

Abstract

In this paper, we investigate the application of multi-resolution maximally stable extremal region (MSER) features for improving the video stabilization performance. MSER features have been used for many computer vision applications like wide baseline stereo, object recognition, video object tracking, and video stabilization with very good results as compared to other features like scale invariant feature transform (SIFT) and Kanade Lucas Tomasi (KLT). However, a limitation of the MSER feature in the stabilization application was observed when the input video frames were severely blurred. The same limitation was also observed when other features like KLT and SIFT were utilized under blurring conditions. In this paper we propose to overcome this drawback for video stabilization application by utilizing MSERs which are extracted and matched in a scale pyramid fashion instead of the MSER features detected and matched on a single image resolution. The duplicate MSERs resulting due to the pyramid style detection are removed followed by MSER feature matching for establishing correspondence between video frames to estimate the global motion parameters. Once the global motion parameters are estimated, the accumulated transformation is smoothed followed by motion compensation to construct the stabilized frame. Comparative analysis with state-of-the-art stabilization methods shows improvement in stabilization performance as well as robustness to blurring degradations. The proposed method can easily be ported to other feature detectors like KLT and SIFT thereby making the proposed method generic to any feature detector.

Extremal Regions Detection Guided by Maxima of Gradient Magnitude

“…MSER has been used in many computer vision applications, like, stereo vision in [18]- [21], object recognition [22]- [24], 3D segmentation [25], lane detection and tracking systems [26], [27], real-time image segmentation [28], finger and hand detection [29], image registration in remote sensing [30], [31], [31], large-scale image retrieval [32], video stabilization [33], and text detection [34]- [36]. Several authors [37]- [39] have proposed improvements on the implementation of MSER.…”