Design of a fault-tolerant middleware for metadata management

Abstract

The Babel File System is a massive storage system, which is made up from three main components: clients, proxy and storage nodes. Each client perceives a single computer, called proxy, which dispatches all service requests randomly to an arbitrary storage node and manages the metadata. However, availability can be compromised if the proxy suddenly halts. To avoid this problem we propose the utilization of a redundant set of proxies. Nevertheless, this solution implies dealing with a consensus problem in order to guarantee the consistency of the metadata copies recorded at each and every proxy. In this paper we propose the use of the Paxos algorithm in order to overcome this problem. Our proposal can be understood as a middleware through which the proxies communicate among themselves to enforce metadata consistency. The experiments showed that, under certain random conditions, the progress of the protocol could be threatened. We propose the mechanisms to address these issues to set the system back to its stable state. However, stability is strongly limited by the correlation between failure and recovery rates.