Configuring large‐scale storage using a middleware with machine learning

Abstract

SUMMARYThe proliferation of cloud services and other forms of service-oriented computing continues to accelerate. Alongside this development is an ever-increasing need for storage within the data centres that host these services. Management applications used by cloud providers to configure their infrastructure should ideally operate in terms of high-level policy goals, and not burden administrators with the details presented by particular instances of storage systems. One common technology used by cloud providers is the Storage Area Network (SAN). Support for seamless scalability is engineered into SAN devices. However, SAN infrastructure has a very large parameter space: their optimal deployment is a difficult challenge, and subsequent management in cloud storage continues to be difficult.In this article, we discuss our work in SAN configuration middleware, which aims to provide users of large-scale storage infrastructure such as cloud providers with tools to assist them in their management and evolution of heterogeneous SAN environments. We propose a middleware rather than a stand-alone tool so that the middleware can be a proxy for interacting with, and informing, a central repository of SAN configurations. Storage system users can have their SAN configurations validated against a knowledge base of best practices that are contained within the central repository. Desensitized information is exported from local management applications to the repository, and the local middleware can subscribe to updates that proactively notify storage users should particular configurations be updated to be considered as sub-optimal, or unsafe.

Middleware strategies for clouds and grids in e‐Science

“…Eyers et al 2 discuss Cloud computing and other large‐scale computing uses needing the support of extensible storage systems in their data centers. However, the complex and heterogeneous infrastructure involved makes it difficult to correctly configure overall systems with the current management tools.…”

“…Relevant contributions have been provided by McEvoy et al 1, Eyers et al 2, Kim et al 3, Pinheiro et al 4, Gomes and Costa 5, Futrelle et al 6, and Ferro et al 7. These contributions focus on: performance and deployment evaluation of a parallel application on a private Cloud; configuring large‐scale storage using a middleware applying machine learning (ML); power‐aware provisioning of virtual machines for real‐time Cloud services; an adaptive fault tolerance mechanism for opportunistic environments with a mobile agent approach; an approach to enhance the efficiency of opportunistic grids; use of semantic content management to create knowledge spaces for e‐Science; and a proposal to apply inductive logic programming (ILP) to self‐healing problem in grids.…”