Advanced Hough-based method for on-device document localization

Abstract

The demand for on-device document recognition systems increases in conjunction with the emergence of more strict privacy and security requirements. In such systems, there is no data transfer from the end device to a third-party information processing servers. The response time is vital to the user experience of on-device document recognition. Combined with the unavailability of discrete GPUs, powerful CPUs, or a large RAM capacity on consumer-grade end devices such as smartphones, the time limitations put significant constraints on the computational complexity of the applied algorithms for on-device execution. In this work, we consider document location in an image without prior knowledge of the docu-ment content or its internal structure. In accordance with the published works, at least 5 systems offer solutions for on-device document location. All these systems use a location method which can be considered Hough-based. The precision of such systems seems to be lower than that of the state-of-the-art solutions which were not designed to account for the limited computational resources. We propose an advanced Hough-based method. In contrast with other approaches, it accounts for the geometric invariants of the central projection model and combines both edge and color features for document boundary detection. The proposed method allowed for the second best result for SmartDoc dataset in terms of precision, surpassed by U-net like neural network. When evaluated on a more challenging MIDV-500 dataset, the proposed algorithm guaranteed the best precision compared to published methods. Our method retained the applicability to on-device computations.

MIDV-2020: a comprehensive benchmark dataset for identity document analysis

“…There exist three base approaches for detecting a quadrangle of the document boundaries: corners detection [34,35], straight lines detection and analysis [36 -39], and the analysis of salient regions [2,40]. As shown in [37], the representatives of all three classes successfully solve the problem given a subset of an open dataset SmartDoc [41], which complies to the problem statement.…”

“…We also asked the authors of [37 -39] to evaluate their algorithms using MIDV-2020. The method described in [37] uses straight lines (for generating quadrangle hypotheses) and salient regions (for comparing and ranging the hypotheses). This algorithm shows state-of-the-art results on the open dataset MIDV-500 [8] and can be used on low-end computational devices.…”

“…This algorithm shows state-of-the-art results on the open dataset MIDV-500 [8] and can be used on low-end computational devices. The algorithms [37] and [39] both require the knowledge of the document's aspect ratio and the knowledge of the camera internal parameters (focal length and position of a principal point). Although we do not know the latter from our problem statement, in experiments with [37] and [39] we used the focal length equal to 0.705 from the diagonal of the image and the position of principal point in the center of the image.…”

“…The algorithms [37] and [39] both require the knowledge of the document's aspect ratio and the knowledge of the camera internal parameters (focal length and position of a principal point). Although we do not know the latter from our problem statement, in experiments with [37] and [39] we used the focal length equal to 0.705 from the diagonal of the image and the position of principal point in the center of the image. The algorithm [38] as well as [37] uses straight lines and salient regions however does not require knowledge of the aspect ratio and internal parameters of the camera.…”

“…6). The link to the source code of the statistics implementation is published in [37]. To perform the numerical evaluation, ground truth document boundary quadrangles were used, which are present in MIDV-2020 for all types of images (except ideal document templates).…”