Text From Corners: A Novel Approach to Detect Text and Caption in Videos

Abstract

Detecting text and caption from videos is important and in great demand for video retrieval, annotation, indexing, and content analysis. In this paper, we present a corner based approach to detect text and caption from videos. This approach is inspired by the observation that there exist dense and orderly presences of corner points in characters, especially in text and caption. We use several discriminative features to describe the text regions formed by the corner points. The usage of these features is in a flexible manner, thus, can be adapted to different applications. Language independence is an important advantage of the proposed method. Moreover, based upon the text features, we further develop a novel algorithm to detect moving captions in videos. In the algorithm, the motion features, extracted by optical flow, are combined with text features to detect the moving caption patterns. The decision tree is adopted to learn the classification criteria. Experiments conducted on a large volume of real video shots demonstrate the efficiency and robustness of our proposed approaches and the real-world system. Our text and caption detection system was recently highlighted in a worldwide multimedia retrieval competition, Star Challenge, by achieving the superior performance with the top ranking.

Comparison of Text Beginning Frame Detection Methods in News Video Sequences

“…The resolution of the videos was 720☓480. The tunable sensitivity parameter k of Harris corner response function is an empirically determined constant from 0.04 to 0.06 [18,19] . In this experiment, the parameter k set to be 0.04.…”

“…Corner points are the image features that are usually more salient and robust than edge for pattern repre- [18] . vertical lines as shown in Fig.…”

Anatomical Survey Based Feature Vector for Text Pattern Detection

“…After applying the morphological dilation on detected corner points in the image, [21] used five region properties as the features to describe text regions. These features are area, saturation, orientation, aspect ratio and position.…”

Bridging text spotting and SLAM with junction features

“…Some corner feature detectors have been shown to be particularly successful for this task, such as Harris corners [21] as used by Zhao et al [22]. An alternative is to consider edges since text is typically composed of strokes with strong gradients on both sides.…”