Transactional filer for grid storage server

Abstract

Future demands on storage systems are expected to stress current storage architectures on all dimensions: amount of aggregate storage is expected to grow by 2 to 3 orders of magnitude, while data-granularity is expected to fall by at least an order of magnitude. Further, the network access bandwidths are also increasing at least by two orders of per link, and several times that in aggregate. We present a Transactional Filer, a extensible scalable file-system capable of "petabyte" and "petabit-per-sec" performance. This filer is built using, as its components, an all-silicon Grid Storage Server, whose scalability derives from a no-software "server-less server" design.The Grid Storage Server implements multiple layers of a protocol stack which are compiled into a hardware engine that processes all layers concurrently on-chip, without any software involvement. The protocol engine chip has a vanishingly low price point, at which it is possible to directly network-enable each individual hard-disk drive. This "serverless" design allows it to scale up to match "terabit" network speeds and "terabyte" disk capacities per node, aggregating to petabyte-petabit capacities across the large scale grid application.The Transactional Filer is a file-system is built upon this distributed silicon protocol engine architecture; and benefits from the advantages of reconfigrable computing. The file-system primitives which are implemented in the hardware protocol engine are simple and hence silicon-friendly, yet they can be combined together in ways to provide a very rich transactional versioned file-system interface to the endapplication. While this newer grid-friendly file-system interface is exposed for new applications, the same hardware primitives can be used in a backward compatible POSIX manner to support legacy applications.