A General-Purpose Architecture for Replicated Metadata Services in Distributed File Systems

Stamatakis, Dimokritos; Tsikoudis, Nikos; Micheli, Eirini C.; Magoutis, Kostas

doi:10.1109/tpds.2017.2700272

Cited by 5 publications

(3 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In all these adaptation mechanisms, the newly assigned read-serving replicas (although consistent at the level of persisted data) may have missed recent reads and thus have an outdated memory cache leading to a performance impact (Section 4). Empirical evidence of the challenge addressed in this paper (and inspiration for this work) is provided by our own previous research [22], [23], [32]. Figure 1 (from [22]) demonstrates that the action of changing the primary replica can hide the performance impact of a backup task, however the improved system (Figure 1b) still suffers from a smaller but non-negligible performance hit (area in red circle) due to cold-cache misses at the new primary (a previously non-read-serving replica).…”

Section: Background and Related Workmentioning

confidence: 99%

“…Figure 1 (from [22]) demonstrates that the action of changing the primary replica can hide the performance impact of a backup task, however the improved system (Figure 1b) still suffers from a smaller but non-negligible performance hit (area in red circle) due to cold-cache misses at the new primary (a previously non-read-serving replica). Other work on measuring the recovery time of a replicated version of the HDFS metadata server [32] ( §4.4.3) showed that switching the primary to a new replica with a cold memory cache can lead to significantly higher time to recover compared to a version switching to a hot spare.…”

Section: Background and Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Addressing the Read-Performance Impact of Reconfigurations in Replicated Key-Value Stores

Papaioannou

Magoutis

2022

IEEE Trans. Parallel Distrib. Syst.

Self Cite

View full text Add to dashboard Cite

Raw data are often orders of magnitude larger than main memory for many applications. As the performance of storage devices is still significantly slower than main memory, systems still rely on memory caching to improve performance. Data replication schemes are prevalent in data stores for high availability and reliability. In such schemes, while data updates are propagated to all replicas (either synchronously or in the background), reads are usually served by only a subset of replica group members (e.g., as in primary-backup and quorum systems). As a result, non-serving replicas cannot keep their memory cache state updated; thus, during a reconfiguration or a fail-over action, the system suffers from a high read-performance impact for a significant amount of time due to cold-cache misses. In our study we observed up to 70% hit after a reconfiguration due to cold cache misses, taking almost 18 minutes in some cases to fully restore to the pre-reconfiguration level of performance.In this paper we propose a mechanism to maintain up-to-date read caches across replicas by sending read hints to the non-serving replicas to keep their caches warm. Thus the system is able to seamlessly achieve the same performance level even in the face of a replica group re-organization. This is especially important under the read-intensive workloads that are common today. Our evaluation shows that our mechanism has significant benefits during reconfigurations, with low performance impact under periods of resource strain. Given its advisory nature, the maintenance of read hints can be reduced or held off if needed during such periods.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Section: Background and Related Workmentioning

confidence: 99%

Addressing the Read-Performance Impact of Reconfigurations in Replicated Key-Value Stores

Papaioannou

Magoutis

2022

IEEE Trans. Parallel Distrib. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In 2017, Stamatakis et al [9] proposed a generalpurpose architecture for High Availability Metadata Services. Metadata is one of the most important components of distributed file system.…”

Section: State Of the Artmentioning

confidence: 99%

A Proposal for High Availability of HDFS Architecture based on Threshold Limit and Saturation Limit of the Namenode

Chakraborty¹,

Barua²,

Pandey³

et al. 2017

IJIEEB

View full text Add to dashboard Cite

Abstract-Big Data which is one of the newest technologies in the present field of science and technology has created an enormous drift of technology to a salient data architecture. The next thing that comes right after big data is Hadoop which has motivated the complete Big Data Environment to its jurisdiction and has reinforced the complete storage and analysis of big data. This paper discusses a hierarchical architecture of Hadoop Nodes namely Namenodes and Datanodes for maintaining a High Availability Hadoop Distributed File System. The High Availability Hadoop Distributed File System architecture establishes itself onto the two fundamental model of Hadoop that is Master-Slave Architecture and elimination of single point node failure. The architecture will be of such utilization that there will be an optimum load on the data nodes and moreover there will be no loss of any data in comparison to the size of data.

show abstract

ICCG: low-cost and efficient consistency with adaptive synchronization for metadata replication

Zhang,

Wang,

Shang

et al. 2024

Front. Comput. Sci.

View full text Add to dashboard Cite

A General-Purpose Architecture for Replicated Metadata Services in Distributed File Systems

Cited by 5 publications

References 28 publications

Addressing the Read-Performance Impact of Reconfigurations in Replicated Key-Value Stores

Addressing the Read-Performance Impact of Reconfigurations in Replicated Key-Value Stores

A Proposal for High Availability of HDFS Architecture based on Threshold Limit and Saturation Limit of the Namenode

ICCG: low-cost and efficient consistency with adaptive synchronization for metadata replication

Contact Info

Product

Resources

About