In this paper, we present PaxosStore, a high-availability storage system developed to support the comprehensive business of WeChat. It employs a combinational design in the storage layer to engage multiple storage engines constructed for different storage models. PaxosStore is characteristic of extracting the Paxos-based distributed consensus protocol as a middleware that is universally accessible to the underlying multi-model storage engines. This facilitates tuning, maintaining, scaling and extending the storage engines. According to our experience in engineering practice, to achieve a practical consistent read/write protocol is far more complex than its theory. To tackle such engineering complexity, we propose a layered design of the Paxos-based storage protocol stack, where PaxosLog, the key data structure used in the protocol, is devised to bridge the programming-oriented consistent read/write to the storage-oriented Paxos procedure. Additionally, we present optimizations based on Paxos that made fault-tolerance more efficient. Discussion throughout the paper primarily focuses on pragmatic solutions that could be insightful for building practical distributed storage systems.

As one of the most popular distributed storage systems, Hadoop Distributed File System (HDFS) has structure to accommodate mass data processing. However, the master/slave architecture only has one single Namenode for metadata management, which has put HDFS in the risk of Single Point of Failure (SPoF). To resolve the problem, there have been a few solutions based on multiple Namenodes. Nevertheless, all of them have some shortcomings in terms of failover efficiency and metadata consistent among Namenodes. Hence, we present a new metadata management schema with two symmetrical Namenodes, which reduces recovery time loss and increases failover efficiency by providing symmetrical read/write service and consistent metadata. To resolve metadata consistency problem, we introduce a third-party node-quorum node, which works together with the other two Namenodes to compose our high availability (HA) architecture Quorum, design a modified triple-machine Paxos algorithm, and finally apply the algorithm to our architecture. Our tests show that the proposed architecture and triple-machine Paxos are feasible.

No abstract