DocScanner: Robust Document Image Rectification with Progressive Learning

Abstract

Compared to flatbed scanners, portable smartphones are much more convenient for physical documents digitizing. However, such digitized documents are often distorted due to uncontrolled physical deformations, camera positions, and illumination variations. To this end, this work presents DocScanner, a new deep network architecture for document image rectification. Different from existing methods, DocScanner addresses this issue by introducing a progressive learning mechanism. Specifically, DocScanner maintains a single estimate of the rectified image, which is progressively corrected with a recurrent architecture. The iterative refinements make DocScanner converge to a robust and superior performance, and the lightweight recurrent architecture ensures the running efficiency. In addition, before the above rectification process, observing the corrupted rectified boundaries existing in prior works, DocScanner exploits a document localization module to explicitly segment the foreground document from the cluttered background environments. To further improve the rectification quality, based on the geometric priori between the distorted and the rectified images, a geometric regularization is introduced during training to further facilitate the performance. Extensive experiments are conducted on the Doc3D dataset and the DocUNet benchmark dataset, and the quantitative and qualitative evaluation results verify the effectiveness of DocScanner, which outperforms previous methods on OCR accuracy, image similarity, and our proposed distortion metric by a considerable margin. Furthermore, our DocScanner shows the highest efficiency in inference time and parameter count.

Geometric Representation Learning for Document Image Rectification

“…However, it involves extra implicit learning to localize the foreground document besides predicting the rectification, which limits the performance. Hence, following [9,10], we adopt a preprocessing operation to remove the clustered background first, thus the following network can focus on the rectification of the distortion.…”

“…Additionally, the 127 th and 128 th distorted images in DocUNet Benchmark dataset [25] are rotated by 180 degrees, which do not match the ground truth documents [10]. It is ignored by existing works [6,7,9,14,22,25,41,42,43].…”

“…As shown in Table 1 in the manuscript, DocGeoNet is efficient in terms of inference time and parameter count compared with existing learning-based methods. One of reasons is that we predict the backward warping flow directly that is used to sample the pixels from the input distorted image for rectification, following [6,9,10]. contrast, previous work DocUNet [25] predicts the forward warping flow instead, which has to be converted to the backward warping flow first using the unstructured points.…”

“…However, the oversimplified parametric models usually lead to limited performance, and the optimization process introduces non-negligible computational cost. Recently, deep learning based solutions [6,7,9,10,19,22,25] have been become a promising alternative to traditional methods. By training a network to directly predict the warping flow, a deformed document image can be rectified by resampling the pixels in the distorted image.…”