Data collection for aggregation of digital repositories: Entities linked to the Special Secretariat of Culture of Brazil

Abstract

This article aims to clarify the use of data science techniques in information science by reporting data collection from the digital repositories of entities linked to the Special Secretariat of Culture of Brazil. This process is part of a research project promoted by the Foundation of Research Support of the State of São Paulo and carried out at the University of Brasília by the Network Intelligence Laboratory. The quantitative descriptive methodology applied alludes to extracting, transforming, and data load used in the constitution of data warehouses, in which Python scripts were developed to collect the data. The results indicate how many scripts were needed and how many storage files were generated, in addition to the description of the data collected, denoting a greater use of the web scraping technique, making the collection process more difficult. Thus, the article points out how the current reality of the analyzed Brazilian cultural institutions is far from enabling the aggregation of their digital repositories. At the same time, it points out how data science strategies allow information science professionals to overcome existing technical barriers and promote data analysis and reuse.