Automatically evaluating the quality of textual descriptions in cultural heritage records

Abstract

Metadata are fundamental for the indexing, browsing and retrieval of cultural heritage resources in repositories, digital libraries and catalogues. In order to be effectively exploited, metadata information has to meet some quality standards, typically defined in the collection usage guidelines. As manually checking the quality of metadata in a repository may not be affordable, especially in large collections, in this paper we specifically address the problem of automatically assessing the quality of metadata, focusing in particular on textual descriptions of cultural heritage items. We describe a novel approach based on machine learning that tackles this problem by framing it as a binary text classification task aimed at evaluating the accuracy of textual descriptions. We report our assessment of different classifiers using a new dataset that we developed, containing more than 100K descriptions. The dataset was extracted from different collections and domains from the Italian digital library “Cultura Italia” and was annotated with accuracy information in terms of compliance with the cataloguing guidelines. The results empirically confirm that our proposed approach can effectively support curators (F1 $$\sim $$ ∼ 0.85) in assessing the quality of the textual descriptions of the records in their collections and provide some insights into how training data, specifically their size and domain, can affect classification performance.

Modelos De Diagnóstico De Qualidade De Dados No Domínio Do Patrimônio Cultural: Uma Revisão Sistemática De Literatura

“…Muitos estudos apresentam alguma deficiência no processo de avaliação, principalmente em relação ao volume de dados avaliados. Em alguns casos, a avaliação foi realizada de forma manual e por amostragem, o que pode resultar em uma análise incompleta e imprecisa, como, por exemplo, nos estudos de Martins e Martins (2021), Moulaison (2015), Palavitsinis (2013), Stephan, Beat e Angelina (2018) e Westbrook et al (2012). Percebe-se aqui ainda a incipiente incorporação de técnicas automatizadas e semiautomatizadas oriundas de áreas como ciência de dados e mais especificamente da aprendizagem de máquina.…”