Towards A Hybrid Approach for Medieval Illuminations Analysis

Abstract

Medieval illuminations represent a system of graphic and visual information, based on metaphorical relationships whose meanings and messages are determined in a specific and evolving context. We consider an illumination as a knowledge graph used by certain elites during the Middle Ages to symbolize their social identity and convey events from their lives, along with their cultural values. This knowledge graph contains a set of entities connected through semantic relationships and was used to encode metaphors and impactful messages that can sometimes be challenging for non-experts to decipher. Our aim is to provide a straightforward approach to decoding those messages. Therefore, we first consider medieval illuminations as images and use YOLOv3 to identify the objects inside a given image using bounding boxes. Moreover, we retrieve the entities from the image, their relationships, and the implicit messages using logical inference. The ultimate aim is to simplify the interpretation of illuminations and provide in the future privacy tools for contemporary social media, such as a logical framework for encoding and transmitting information.