We present a power-saving method for largescale distributed storage systems. The key idea is to use virtual nodes and migrate them dynamically so as to skew the workload towards a small number of disks while not overloading them. Our proposed method consists of two kinds of algorithms, one for gathering or spreading virtual nodes according to the daily variation of workloads so that the active disks are reduced to a minimum, the other for coping with the changes in the popularity of data over a longer period. For this dynamic migration, data stored in virtual nodes are managed by a distributed hash table. Furthermore, to improve the reliability as well as to reduce the migration cost, we also propose an extension of our method by introducing a replication mechanism. The performance of our method is measured both by simulation and a prototype implementation. From the experiments, we observed that our method skews the workload so that the average load for the active physical nodes as a function of the overall capacity is 67%. At the same time, we maintain a preferred response time by setting a suitable maximum workload for each physical node.

No abstract

No abstract

The growing amount of data and advances in data science have created a need for a new kind of cloud platform that provides users with flexibility, strong security, and the ability to couple with supercomputers and edge devices through highperformance networks. We have built such a nation-wide cloud platform, called "mdx" to meet this need. The mdx platform's virtualization service, jointly operated by 9 national universities and 2 national research institutes in Japan, launched in 2021, and more features are in development. Currently mdx is used by researchers in a wide variety of domains, including materials informatics, geo-spatial information science, life science, astronomical science, economics, social science, and computer science. This paper provides an the overview of the mdx platform, details the motivation for its development, reports its current status, and outlines its future plans.