Model building for dynamic multi-tenant provider environments

Abstract

Increasingly, storage vendors are finding it difficult to leverage existing white-box and black-box modeling techniques to build robust system models that can predict system behavior in the emerging dynamic and multi-tenant data centers. White-box models are becoming brittle because the model builders are not able to keep up with the innovations in the storage system stack, and black-box models are becoming brittle because it is increasingly difficult to a priori train the model for the dynamic and multi-tenant data center environment. Thus, there is a need for innovation in system model building area. In this paper we present a machine learning based blackbox modeling algorithm called M-LISP that can predict system behavior in untrained region for these emerging multitenant and dynamic data center environments. We have implemented and analyzed M-LISP in real environments and the initial results look very promising. We also provide a survey of some common machine learning algorithms and how they fare with respect to satisfying the modeling needs of the new data center environments.

Queueing-based storage performance modeling and placement in OpenStack environments

“…However, as the IO load increases, the response time exhibits nonlinear behavior and the device reaches to saturation stage quickly, i.e., the response time becomes intolerable for many applications. Similar hockey-shaped storage performance curves have been observed and validated extensively in the literature via comprehensive benchmarking experiments, e.g., in [4], [5]. From a performance modeling perspective, if the performance curve of each storage device is known, either via benchmarking or provided by the storage device manufacturer, the storage administrator can easily estimate the response time changes for better storage management.…”

“…The regression decision tree is trained using historical workload and performance measurement data. The CART model is further extended in [15] to estimate the saturation throughput of a storage device by incorporating additional parameters such as outstanding IO requests, and in [4], [16] to incorporate advanced machine learning methods to further reduce the estimation errors. Similar regression tree based methods have been applied to create the performance models of SSD devices in [17], for HPC application performance prediction in [18], and the relative fitness models between pairs of storage devices in [19].…”

Towards Lightweight and Swift Storage Resource Management in Big Data Cloud Era

“…Amazon AWS [16], Microsoft Azure [17]) and run big data applications [18], workload IO behavior can heavily fluctuate even within one day. Basak et al [7] built a dynamic performance model for multi-tenant cloud. However, no resource-scheduling algorithm is proposed.…”