Combining Attention Module and Pixel Shuffle for License Plate Super-Resolution

Super-resolution of license plate images using attention modules and sub-pixel convolution layers

“…We present a novel approach for improving LP superresolution through the use of PixelShuffle (PS) layers and a Three-Fold Attention Module. Our method extends the work of Mehri et al [15] and Nascimento et al [16] by taking into account not only the pixel intensity values, but also structural and textural information. To further enhance the performance, we incorporate an auto-encoder that extracts shallow features by squeezing and expanding the network constructed with PS and PixelUnshuffle (PU) layers.…”

“…This section details our super-resolution approach that enhances the extraction of structural and textural features from low-resolution LPs. Our network extends the network proposed in our previous work [16], further expanding the MPRNet architecture and TFAM algorithm by Mehri et al [15] while taking inspiration from [36] to improve the proposed attention module to enable the network to capture structural and textural information. The proposed approach leverages a novel perceptual loss function that uses an OCR model as a feature extractor.…”

“…In Fig. 2, we show our modifications to the MPRNet's TFAM [15] and to the attention module by Nascimento et al [16] to create the PLTFAM. The design of this module is based on the following insights: (i) images are composed of the relationship between channels, where each channel contributes unique characteristics to form the final image, therefore, the extraction of these features is crucial for proper image restoration; (ii) the positional information of these essential features from the channels composing the images is required; (iii) traditional downscale and upscale operations rely on translational invariance and interpolation techniques, which are not able to learn a custom process for different tasks; (iv) the module captures salient structure from the character fonts of the LP, highlighting both structure and textural features in the image.…”

“…• A super-resolution approach that builds upon MPR-Net [15] and the architecture we proposed in [16] (see the next paragraph) by incorporating subpixel-convolution layers (PS and PU) in combination with a Pixel Level Three-Fold Attention Module (PLTFAM); • A novel perceptual loss that combines features extracted by an OCR model [17] with L1 loss to reconstruct characters with the most relevant characteristics. This loss function allows the use of any OCR model for LPR; • The datasets we built for this work, as well as the source code, are publicly available to the research community.…”

“…This loss function allows the use of any OCR model for LPR; • The datasets we built for this work, as well as the source code, are publicly available to the research community. A preliminary version of this study was presented at the 2022 Conference on Graphics, Patterns and Images (SIB-GRAPI) [16]. The approach described here differs from the previous version in several aspects.…”