Self-supervised deep reconstruction of mixed strip-shredded text documents

Abstract

The reconstruction of shredded documents consists of coherently arranging fragments of paper (shreds) to recover the original document(s). A great challenge in computational reconstruction is to properly evaluate the compatibility between the shreds. While traditional pixel-based approaches are not robust to real shredding, more sophisticated solutions compromise significantly time performance. The solution presented in this work extends our previous deep learning method for single-page reconstruction to a more realistic/complex scenario: the reconstruction of several mixed shredded documents at once. In our approach, the compatibility evaluation is modeled as a two-class (valid or invalid) pattern recognition problem. The model is trained in a self-supervised manner on samples extracted from simulated-shredded documents, which obviates manual annotation. Experimental results on three datasets-including a new collection of 100 strip-shredded documents produced for this work-have shown that the proposed method outperforms the competing ones on complex scenarios, achieving accuracy superior to 90%.

Unveiling the Secrets: Reconstruction of Shredded Documents using Deep Learning

“…• A deep classification-based approach [12], [13] (Section II). The problem of evaluating compatibility is formulated as a classification problem using a deep model: the obtained results demonstrate the effectiveness of this approach in reconstructing 100 documents with an accuracy superior to 90%; • A deep metric-learning approach [14] (Section III).…”

High-efficient low-cost characterization of materials properties using domain-knowledge-guided self-supervised learning

“…In contrast, as a new paradigm between unsupervised and supervised learning, SSL can generate labels based on the property of unlabeled data itself to train the neural network in a supervised manner similar to natural learning experiences. With excellent performance on representation learning and dealing with the issue of unlabelled data, SSL [20][21][22] has been successfully implemented in a wide range of fields, including image recognition 23 , audio representation 24 , computer vision 25 , document reconstruction 26 , atmosphere 27 , astronomy 28 , medical 29 , person re-identification 30 , remote sensing 31 , robotics 32 , omnidirectional imaging 33 , manufacturing 34 , nano-photonics 35 , and civil engineering 36 , etc. However, this method has not been formally attempted in material science.…”

Self-supervised deep metric learning for ancient papyrus fragments retrieval

“…In document information retrieval, the authors of [4] propose an interesting and promising approach using clustered SIFT descriptors ( [25]) as pretext-task labels for writer retrieval. The authors of [33] generate simulated-shredded documents to train a model for reconstructing mixed strip-shredded text documents. An other interesting approach is proposed in [9], where the authors apply various random transformations to the unlabelled source images to create a variety of images that compose a surrogate class, then used for supervised training.…”