PGNet: Real-time Arbitrarily-Shaped Text Spotting with Point Gathering Network

Abstract

The reading of arbitrarily-shaped text has received increasing research attention. However, existing text spotters are mostly built on two-stage frameworks or character-based methods, which suffer from either Non-Maximum Suppression (NMS), Region-of-Interest (RoI) operations, or character-level annotations. In this paper, to address the above problems, we propose a novel fully convolutional Point Gathering Network (PGNet) for reading arbitrarily-shaped text in real-time. The PGNet is a single-shot text spotter, where the pixel-level character classification map is learned with proposed PG-CTC loss avoiding the usage of character-level annotations. With a PG-CTC decoder, we gather high-level character classification vectors from two-dimensional space and decode them into text symbols without NMS and RoI operations involved, which guarantees high efficiency. Additionally, reasoning the relations between each character and its neighbors, a graph refinement module (GRM) is proposed to optimize the coarse recognition and improve the end-to-end performance. Experiments prove that the proposed method achieves competitive accuracy, meanwhile significantly improving the running speed. In particular, in Total-Text, it runs at 46.7 FPS, surpassing the previous spotters with a large margin.

Preface: Celebrating the 70th Anniversary of School of Mechanical Science and Engineering, Huazhong University of Science and Technology

“…In the past few years, many methods attempt to join text detection and recognition by proposing a new Region-of-Interest (RoI) operation to achieve the synergy between text detection and text recognition [32,12,50,33,52], as shown in Figure 1(a). They follow the classical two-stage pipeline, which first locates the text instance and then extracts the text content in the corresponding region of interest (RoI).…”