Reinforcement Learning for Smart Caching at the CMS experiment

Tedeschi, T.; Tracolli, Mirco; Ciangottini, D.; Storchi, Loriano; Baioletti, Marco; Poggioni, Valentina

doi:10.22323/1.378.0009

Cited by 3 publications

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Contrasting with existing efforts within the WLCG community, such as those employing Reinforcement Learning for caching tasks ( [13]), our approach adopts the RF model. We favoured RF due to its ability to effectively handle both classification and regression tasks, along with its capability to be parallelized.…”

Section: Ml-based Solution 41 Architecturementioning

confidence: 99%

Data Popularity for Cache Eviction Algorithms using Random Forests

Chuchuk,

Schulz

2024

EPJ Web of Conf.

View full text Add to dashboard Cite

In the HEP community the prediction of Data Popularity is a topic that has been approached for many years. Nonetheless, while facing increasing data storage challenges, especially in the upcoming HL-LHC era, there is still the need for better predictive models to answer the questions of whether particular data should be kept, replicated, or deleted. Caches have proven to be a convenient technique for partially automating storage management, potentially eliminating some of these questions. On the one hand, one can benefit even from simple cache eviction policies like LRU, on the other hand, we show that incorporation of knowledge about future access patterns has the potential to greatly improve cache performance. In this paper, we study data popularity on the file level, where the special relation between files belonging to the same dataset could be used in addition to the standard attributes. We turn to Machine Learning algorithms, such as Random Forest, which is well suited to work with Big Data: it can be parallelized, is more lightweight and easier to interpret than Deep Neural Networks. Finally, we compare the results with standard cache eviction algorithms and the theoretical optimum.

show abstract

Section: Ml-based Solution 41 Architecturementioning

confidence: 99%

Data Popularity for Cache Eviction Algorithms using Random Forests

Chuchuk,

Schulz

2024

EPJ Web of Conf.

View full text Add to dashboard Cite

show abstract

Smart Caching in a Data Lake for High Energy Physics Analysis

et al. 2023

View full text Add to dashboard Cite

The continuous growth of data production in almost all scientific areas raises new problems in data access and management, especially in a scenario where the end-users, as well as the resources that they can access, are worldwide distributed. This work is focused on the data caching management in a Data Lake infrastructure in the context of the High Energy Physics field. We are proposing an autonomous method, based on Reinforcement Learning techniques, to improve the user experience and to contain the maintenance costs of the infrastructure.

show abstract