Cluttered TextSpotter: An End-to-End Trainable Light-Weight Scene Text Spotter for Cluttered Environment

Abstract

Scene text spotting aims at simultaneously localizing and recognizing text instances, symbols, and logos in natural scene images. Scene text detection and recognition approaches have received immense attention in computer vision research community. The presence of partial occlusion or truncation artifact due to the cluttered background of scene images creates an obstacle in perceiving the text instances, which makes the process of spotting very complex. In this paper, we propose a lightweight scene text spotter that can address the issue of cluttered environment of scene images. It is an end-to-end trainable deep neural network that uses local part information, global structural features, and context cue information of oriented region proposals for spotting text instances. It helps to localize in scene images with background clutters, where partially occluded text parts, truncation artifacts, and perspective distortions are present. We mitigate the problem of misclassification caused by inter-class interference by exploring inter-class separability and intra-class compactness. We also incorporate multi-language character segmentation and word-level recognition in a lightweight recognition module. We have used six publicly available benchmark datasets in different smart devices to illustrate the efficacy of the network.

Mining text from natural scene and video images: A survey

“…This approach largely depends on the results of the character detection step and therefore, accumulated errors are the major concern in this approach. In word‐based text recognition methods, each word is considered as a whole and holistic word classification is commonly performed to achieve word recognition (Bagi et al, 2020a). A dictionary of segmented words may further need to be considered in this approach.…”

“…A special backbone for text features and two different types of attention have been considered to achieve state‐of‐the‐art performance for both text spotting and script identification in natural images (Y. Zhou, Fang, et al, 2019). Recently, Bagi et al (2020a) have proposed an end‐to‐end trainable deep neural network based on local, global, and contextual information of multiscale feature maps of a lightweight backbone network for text spotting instances in scene images with background clutters, partially occluded text, truncation artifacts, and perspective distortions. The problem of inter‐class misclassification has been addressed by maximizing inter‐class separability and compacting intra‐class variability using Gaussian softmax.…”

“…Multilanguage character segmentation and word‐level recognition have also been incorporated into the system. The proposed text spotting method provided high accuracies for detecting multilingual text, logos, and symbols in scene images with the cluttered background environment captured from resource‐constrained devices, such as smartphones (Bagi et al, 2020a). Furthermore, Y. Liu, Chen, Du, and Tao (2020)have introduced an end‐to‐end trainable unified framework for arbitrary shape text spotting by integrating holistic level, pixel level, and sequence level semantic information into the system.…”

“…It is also 10 times faster than recent state‐of‐the‐art methods (J. Liu, Zhong, Yuan, et al, 2020). To efficiently handle text spotting in blurry scene images, Bagi et al (2020a) has proposed a text spotter called Blurred TextSpotter. An encoder–decoder, as the backbone network, based on multiscale contextual information followed by spatial and channel‐wise attention have been considered in the Blurred TextSpotter.…”

“…As an example, an intelligent transportation system can significantly transform the traffic and travel experience of people. Driver assistance systems and autonomous cars are crucial parts of such a system that improve the safety and security of passengers (Bagi et al, 2020a, Bagi et al, 2020b).…”