Self-supervised deep metric learning for ancient papyrus fragments retrieval

Abstract

This work focuses on document fragments association using Deep Metric Learning methods. More precisely, we are interested in ancient papyri fragments that need to be reconstructed prior to their analysis by papyrologists. This is a challenging task to automatize using machine learning algorithms because labeled data is rare, often incomplete, imbalanced and of inconsistent conservation states. However, there is a real need for such software in the papyrology community as the process of reconstructing the papyri by hand is extremely time consuming and tedious. In this paper, we explore ways in which papyrologists can obtain useful matching suggestion on new data using Deep Convolutional Siamese-Networks. We emphasize on low-to-no human intervention for annotating images. We show that the from-scratch self-supervised approach we propose is more effective than using knowledge transfer from a large dataset, the former achieving a top-1 accuracy score of 0.73 on a retrieval task involving 800 fragments.The research leading to this results has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program under grant agreement No 758907 and is part of the GESHAEM project, hosted by the Ausonius Institute. The source code (upon request) and data used in this article is available at https://morphoboid.labri.fr/ self-supervised-papyrus.html

Feature Mixing for Writer Retrieval and Identification on Papyri Fragments

“…Writer Retrieval and Identification for Papyri. Pirrone et al [11] investigate a self-supervised approach for retrieving papyri fragments using a Siamese network with contrastive loss. They evaluate their method on the Michigan Papyrus Collection and a subset of HisFragIR20.…”

Deep Learning-based Reconstruction of Shredded Documents

“…Although focus is on the reconstruction application, the proposed techniques can be extended to other related applications: e.g. solving jigsaw puzzles with eroded borders [Paumard et al 2020, Li et al 2021, Rika et al 2022 and reconstruction of ancient papyrus [Abitbol et al 2021, Pirrone et al 2021.…”