Unveiling the Past: A Holistic Approach to Rescuing Historical Texts through Advanced Image Analysis and AI

Abstract

In recent years, more and more administrative and legal paperwork has been done using computers because it's efficient. But in the past, when technology wasn't as good, there were problems with printing documents. This meant that text could be hard to read, the ink might not be consistent, and documents could get damaged over time. This caused a lot of important information, especially in ancient scripts, to be lost because it couldn't be digitised. Our research is all about solving this problem. We want to find a way to automatically get accurate information from pictures of these old documents. We employ a multi-step approach to enhance text recovery. Firstly, we utilise adaptive Gaussian thresholding to improve image clarity by removing excess ink or stains. Next, we apply optical character recognition (OCR) using advanced systems like Easy OCR and Tesseract, followed by thorough database validation for result accuracy. But sometimes, even with the special software, we can't see all the words because they're too faded or missing. To overcome this, we employ Natural Language Processing (NLP) techniques with Happy Transformer, a tool based on Hugging Face's Transformer Library. This enables us to predict and reconstruct missing letters or words. After prediction and cross-referencing with the database, the identified words are seamlessly integrated into the text, ensuring data retrieval. Our research is important for preserving historical and legal documents that might otherwise be lost forever.