Strategies for Efficiently Keeping Local Linked Open Data Caches Up-To-Date

Abstract

Quite often, Linked Open Data (LOD) applications pre-fetch data from the Web and store local copies of it in a cache for faster access at runtime. Yet, recent investigations have shown that data published and interlinked on the LOD cloud is subject to frequent changes. As the data in the cloud changes, local copies of the data need to be updated. However, due to limitations of the available computational resources (e.g., network bandwidth for fetching data, computation time) LOD applications may not be able to permanently visit all of the LOD sources at brief intervals in order to check for changes. These limitations imply the need to prioritize which data sources should be considered first for retrieving their data and synchronizing the local copy with the original data. In order to make best use of the resources available, it is vital to choose a good scheduling strategy to know when to fetch data of which data source. In this paper, we investigate different strategies proposed in the literature and evaluate them on a large-scale LOD dataset that is obtained from the LOD cloud by weekly crawls over the course of three years. We investigate two different setups: (i) in the single step setup, we evaluate the quality of update strategies for a single and isolated update of a local data cache, while (ii) the iterative progression setup involves measuring the quality of the local data cache when considering iterative updates over a longer period of time. Our evaluation indicates the effectiveness of each strategy for updating local copies of LOD sources, i. e, we demonstrate for given limitations of bandwidth, the strategies' performance in terms of data accuracy and freshness. The evaluation shows that the measures capturing change behavior of LOD sources over time are most suitable for conducting updates.

Scheduling Refresh Queries for Keeping Results from a SPARQL Endpoint Up-to-Date (Short Paper)

“…In order to reduce this overhead we investigate effective scheduling strategies in this paper. A similar investigation in the context of updates of Linked Data has been presented by Dividino et al [1]. The authors show that change-aware strategies are suitable to keep local data caches up-to-date.…”

“…Dynamics-Jaccard (DJ) has been proposed as a best-effort scheduling policy for dataset updates [1]. Here, for DESCRIBE and CONSTRUCT queries we compute the Jaccard distance on RDF triples, and on the query solutions for SELECT queries.…”

Analysing the Evolution of Knowledge Graphs for the Purpose of Change Verification

“…For the other RDF documents, the most dynamic elements are literals. Dividino et al [18] analyzed the dynamics of KGs and have proposed a monotone, non-negative function to represent the dynamics of a set of triples as a single numerical value. We revealed representative patterns of entity changes using the DyLDO dataset [19].…”

A Dynamic, Cost-Aware, Optimized Maintenance Policy for Interactive Exploration of Linked Data

“…Analytics applications need constant updates to guarantee the quality of service in maintaining the local cache. To the extent of our knowledge, there is limited work addressing the problem of maintaining local views (or caches) up-to-date [6,[10][11][12][13][14][15][16]. Normally a maintenance policy is needed to determine what to update and when.…”