Rectification and Super-Resolution Enhancements for Forensic Text Recognition

Blanco‐Medina, Pablo; Fidalgo, Eduardo; Alegre, Enrique; Aláiz-Rodríguez, Rocío; Jáñez-Martino, Francisco; Bonnici, Alexandra

doi:10.3390/s20205850

Cited by 7 publications

(5 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The critical challenge of applying the text extraction technique to darknet market photos is raised by the image quality, as the resolution of images is often quite low. After detecting the text area within the picture, in order to resolve the problem, we apply a super-resolution technique, which improves the image quality [35]. The text data is then recognized and extracted from the improved image.…”

Section: Textmentioning

confidence: 99%

“…Third, text data is extracted from the product images (if there are any) to figure out additional features that cannot be captured using the image hash. However, as most of the darknet market photos have low image quality in practice, high accuracy cannot be achieved by simply applying the text extraction tool to darknet market photos [35]. We thus apply a super-resolution [37] method to improve the performance of text recognition [36,38].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detect Your Fingerprint in Your Photographs: Photography-based Multi-Feature Sybil Detection

Kim¹,

Kim²,

Chung³

et al. 2023

PoPETs

View full text Add to dashboard Cite

A darknet market is an online marketplace typically implemented over Tor, where vendors sell illegal products or criminal services. Due to dramatic growth in the popularity of such markets, there is a recognized need for automatic investigation of the market’s ecosystem and identification of anonymous vendors. However, as they often create multiple accounts (or Sybil accounts) within or across different marketplaces, detecting Sybil accounts becomes the key to understanding the ecosystem of darknet markets and identifying the actual relationship between the vendors. This study presents a novel Sybil detection method that extracts multiple features of vendors from photographs in a fine-grained level (e.g., image similarity, main category, subcategory, and text data), and reveals the multiple Sybil accounts of them simultaneously. Each feature is extracted from multiple rich sources using an image hash algorithm, Deep Neural Network (DNN) classifier, image restoration, and text recognition tool; and merged using a weighted feature embedding model. The matching score of each vendor is then calculated to identify not only the exact Sybil accounts, but multiple potential accounts suspected of being associated to a single operator. We evaluate the efficacy of our method using real-world datasets from four large darknet markets (i.e., SilkRoad2, Agora, Evolution, Alphabay) from 2014 to 2015. Because of the anonymity of darknet market, we construct the ground-truth of Sybil accounts by randomly splitting the dataset of vendors into two even parts. We used the first set to train the model, and linked the second set to the original vendor in the first set to evaluate performance. Our experimental results demonstrated that the proposed method outperforms the existing photography-based system with an accuracy of 98%, identifying up to 700% more candidate Sybil accounts than prior work. Additionally, our method detects multiple Sybil accounts for 90% of evaluated test cases, presenting a very different picture of darknet marketplace dynamics than methods that can only detect a single Sybil account for each target vendor. Due to its fine-grained multiple feature extraction from photographs, our method can be more generically applied to various darknet markets for Sybil detection regardless of their languages or categories of items.

show abstract

Section: Textmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Detect Your Fingerprint in Your Photographs: Photography-based Multi-Feature Sybil Detection

Kim¹,

Kim²,

Chung³

et al. 2023

PoPETs

View full text Add to dashboard Cite

show abstract

“…When using dictionaries, the total edit distance can also be reported as a performance measure, although it is not commonly used across most algorithms. This metric compares the labelled string with the transcription, giving a numerical value to the distance between both words for each of the images within the test set [17, 22, 35].…”

Section: Datasets and Performancementioning

confidence: 99%

“…Due to these problems, properly comparing the different methods can be an arduous task [15]. Moreover, searching for a solution that could be integrated into a real tool or service [16, 17] with specific requirements in particular environments (e.g., forensic software applications) may represent a challenge for researchers [18].…”

Section: Introductionmentioning

confidence: 99%

A survey on methods, datasets and implementations for scene text spotting

Blanco‐Medina

Fidalgo

Alegre

et al. 2022

IET Image Processing

Self Cite

View full text Add to dashboard Cite

Text Spotting is the union of the tasks of detection and transcription of the text that is present in images. Due to the various problems often found when retrieving text, such as orientation, aspect ratio, vertical text or multiple languages in the same image, this can be a challenging task. In this paper, the most recent methods and publications in this field are analysed and compared. Apart from presenting features already seen in other surveys, such as their architectures and performance on different datasets, novel perspectives for comparison are also included, such as the hardware, software, backbone architectures, main problems to solve, or programming languages of the algorithms. The review highlights information often omitted in other studies, providing a better understanding of the current state of research in Text Spotting, from 2016 to 2022, current problems and future trends, as well as establishing a baseline for future methods development, comparison of results and serving as guideline for choosing the most appropriate method to solve a particular problem.

show abstract

“…As a subfield of SISR, the purpose of scene text image superresolution (STISR) is to improve the visual quality of LR text images by accurately reconstructing the blurry and illegible characters. It would be promising to introduce STISR methods as a preprocessor to enhance text recognition [14,21]. STISR has been an active research topic recently.…”

Section: Introductionmentioning

confidence: 99%

TLWSR: Weakly supervised real‐world scene text image super‐resolution using text label

Shi

Zhu

Fang

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

Scene text image super-resolution (STISR) has recently received considerable attention. Existing STISR methods are applicable to the situation that all the LR-HR pairs are available. However, in real-world scenarios, it is difficult and expensive to collect ground-truth HR labels and align them with LR images, and thus it is essential to find a way to implement weakly supervised learning. We investigate the STISR problem in the situation that only a subset of HR labels is available and design a weak supervision framework using coarsegrained text labels named TLWSR, which combines incomplete supervision and inexact supervision. Specifically, a lightweight text recognition network and connectionist temporal classification loss are used to guide the super-resolution of text images during training. Extensive experiments on the benchmark TextZoom demonstrate that TLWSR generates distinguishable text images and exceeds the fully supervised baseline TSRN in boosting text recognition accuracywith only 50% HR labels available. Meanwhile, TLWSR can be applied to different super-resolution backbones and significantly improves their performance. Furthermore, TLWSR shows good generalization capability to low-quality images on scene text recognition benchmarks, which verifies the effectiveness of this framework. To the authors' knowledge, this is the first work exploring the problem of STISR in weakly supervised scenarios.

show abstract

Rectification and Super-Resolution Enhancements for Forensic Text Recognition

Cited by 7 publications

References 41 publications

Detect Your Fingerprint in Your Photographs: Photography-based Multi-Feature Sybil Detection

Detect Your Fingerprint in Your Photographs: Photography-based Multi-Feature Sybil Detection

A survey on methods, datasets and implementations for scene text spotting

TLWSR: Weakly supervised real‐world scene text image super‐resolution using text label

Contact Info

Product

Resources

About