What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labels

Abstract

Scene text recognition (STR) task has a common practice: All state-of-the-art STR models are trained on large synthetic data. In contrast to this practice, training STR models only on fewer real labels (STR with fewer labels) is important when we have to train STR models without synthetic data: for handwritten or artistic texts that are difficult to generate synthetically and for languages other than English for which we do not always have synthetic data. However, there has been implicit common knowledge that training STR models on real data is nearly impossible because real data is insufficient. We consider that this common knowledge has obstructed the study of STR with fewer labels. In this work, we would like to reactivate STR with fewer labels by disproving the common knowledge. We consolidate recently accumulated public real data and show that we can train STR models satisfactorily only with real labeled data. Subsequently, we find simple data augmentation to fully exploit real data. Furthermore, we improve the models by collecting unlabeled data and introducing semi-and self-supervised methods. As a result, we obtain a competitive model to state-of-the-art methods. To the best of our knowledge, this is the first study that 1) shows sufficient performance by only using real labels and 2) introduces semi-and self-supervised methods into STR with fewer labels. Our code and data are available: https: //github.com/ku21fan/STR-Fewer-Labels.Recently, public real data has been sufficiently accumulated to train STR models, as shown in Figure 1. We consolidate the training set of public real datasets from 2011 to 2019. Table 1 lists datasets. Figure 3 shows the examples of word boxes. Before using the original data directly for training, we conduct some preprocessing on datasets for our task. We summarize the processes in §3.3, and details are in the supplementary materials.

SGBANet: Semantic GAN and Balanced Attention Network for Arbitrarily Oriented Scene Text Recognition

“…The existing methods [3,4] do not recognize the characters correctly, while the proposed method reports correct recognition results. In addition, We have calculated the average FPS of the proposed and existing methods [3,4] for all the 8 datasets and the results are 6.76, 6.68 and 7.58 for the methods [3], [4] and SGBANet, respectively. This shows that our method is faster than the existing methods.…”

“…It can be observed from Fig. 1 that the existing methods [4,11] do not recognize the characters correctly for arbitrarily shaped text and text with complex backgrounds. Therefore, designing a robust method for recognizing arbitrarily shaped text is still a challenging task that remains to be solved.…”

Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Study

“…As the thriving of deep learning, the researchers also made attempts to build the text recognition models based on deep neural networks following the bottom-up fashion [2,3,5,6,22,23,29,35,38,[40][41][42][43]46,47,51,59,60,65,67,71,73,83,87,90,98,100]. For example, CRNN [59] utilizes the CNN-RNN architecture to extract features for the text images, which are further supervised with the CTC loss [24] to maximize the probability of the ground truth.…”

Perceiving Stroke-Semantic Context: Hierarchical Contrastive Learning for Robust Scene Text Recognition

“…In this work, we adopt STR public datasets to evaluate the performance of the pre-trained model. The datasets cover A New Dataset: UTI-100M As suggested in literature (Baek, Matsui, and Aizawa 2021), training model on real data can yield better results than synthetic data. Therefore, we collect a large-scale real dataset containing about 100 million unlabeled text line images, named Unlabeled Text Image 100M (UTI-100M), to explore the potential of the proposed hierarchical contrastive learning paradigm.…”