Storage workload modelling by hidden Markov models: Application to Flash memory

Harrison, Peter G.; Harrison, Sarah K.; Patel, Niki; Zertal, Soraya

doi:10.1016/j.peva.2011.07.022

Cited by 15 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This assumption has been tested in our Multi-User classification model. It has already been established that HMMs, combined with the supporting clustering analysis and appropriate choice of bins, are able to provide a concise, parsimonious and portable synthetic workload [10]. However, the deficiency of such models is their heavy computing resource requirement, which essentially precludes them from any form of parallel or Multi-User analysis.…”

Section: Discussionmentioning

confidence: 99%

“…The underlying properties of the HMM (including its stochastic, predictive and parsimonious nature) have made it appealing in classifying user profiles on social media. These defining characteristics of the HMM have led to a range of applications in a wide variety of fields: originally, HMMs were used in Speech Recognition [9], [16] and Genome Sequence Prediction [12]; more recently, they have represented Storage Workloads [10], [20] and have modeled Hospital Patient Arrivals [20] as well as Social Network Interactions [19]. In all cases, these models have employed the well-known statistical algorithms to solve three fundamental problems credited to HMMs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Modeling Multi-user Behaviour in Social Networks

Chis

Harrison

2014

2014 IEEE 22nd International Symposium on Modelling, Analysis &Amp; Simulation of Computer and Telecommunication Systems

View full text Add to dashboard Cite

Social networks, and the behaviour of groups of online users, are popular topics in modeling and classifiyng Internet traffic data. There is a need to analyze online network performance metrics through suitable workload benchmarks. We address this issue with a Multi-dimensional Hidden Markov Model (MultiHMM) to act as a Multi-User workload classifier. The MultiHMM is an adaptation of the original HMM, using clustering methods and multiple trace-training for the BaumWelch algorithm. The goals of the MultiHMM are to classify multiple online user streams with minimal processing needs, represent burstiness and correlation among groups of users and to improve security measures in the social network. Experiments are carried out using multiple traces from Twitter data, where original traces are analysed and compared with the MultiHMMgenerated traces. The metrics involved in validating our model include means, standard deviations, skewness and autocorrelation, and we discuss applications and extensions of our model.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Modeling Multi-user Behaviour in Social Networks

Chis

Harrison

2014

2014 IEEE 22nd International Symposium on Modelling, Analysis &Amp; Simulation of Computer and Telecommunication Systems

View full text Add to dashboard Cite

show abstract

“…They are synthetic but based on the characteristics of real extracted traces, then they are in accordance with the real traces and preserve all their key characteristics. The representativeness is ensured using HMM (Hidden Markov Model) for OLTP [7] and I/O characterization works achieved by Park et al [15]. The workloads parameters are summarized below: We noticed regular I/O request sizes for the OLTP and less regular ones for the scientific (physics) applications.…”

Section: Workloadsmentioning

confidence: 95%

OGSSim: Open Generic data Storage systems Simulation tool

Gougeaud¹,

Zertal²,

Lafoucriere³

et al. 2015

Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques

View full text Add to dashboard Cite

In this paper, an open and generic storage simulator is proposed. It simulates with accuracy multi-tiered storage systems based on heterogeneous devices including HDDs, SSDs and the connecting buses. The target simulated system is constructed from the hardware configuration input, then sent to the simulator modules along with the trace file and the appropriate simulator functions are selected and executed. Each module of the simulator is executed by a thread, and communicates with the others via ZeroMQ, a message transmission API using sockets for the information transfer. The result is an accurate behavior of the simulated system submitted to a specific workload and represented by performance and reliability metrics. No restriction is put on the input hardware configuration which can handle different levels of details and makes this simulator generic. The diversity of the supported devices, regardless to their nature: disks, buses, ..etc and organisation: JBOD, RAID, ..etc makes the simulator open to many technologies. The modularity of its design and the independence of its execution functions, makes it open to handle any additional mapping, access, maintenance or reconstruction strategies. The conducted tests using OLTP and scientific workloads show accurate results, obtained in a competitive runtime.

show abstract

“…resource utilization [26] and power consumption [27]. The modeling approach provides a foundational methodology to abstract and represents the relationship between a particular actor and the target testing objective.…”

Section: Performance Modelingmentioning

confidence: 99%

Transfer Learning Based Performance Modeling And Effective Storage Management In Big Data Ecosystems

Alsayoud¹

2021

Preprint

View full text Add to dashboard Cite

Big data ecosystems contain a mix of sophisticated hardware storage components to support heterogeneous workloads. Storage components and the workloads interact and affect each other; therefore, their relationship has to consider when modeling workloads or managing storage. Efficient workload modeling guides optimal storage management decisions, and the right decisions help guarantee the workload’s needs. The first part of this thesis focuses on workload modeling efficiency, and the second part focuses on cost-effective storage management.<div>Workload performance modeling is an essential step in management decisions. The standard modeling approach constructs the model based on a historical dataset collected from one set of setups (scenario). The standard modeling approach requires the model to be reconstructed from scratch with every time the setups changes. To address this issue, we propose a cross-scenario modeling approach that improves the workload’s performance classification accuracy by up to 78% through adopting the Transfer Learning (TL).<br></div><div>The storage system is the most crucial component of the big data ecosystem, where the workload’s execution process starts by fetching data from it and ends by storing data into it. Thus, the workload’s performance is directly affected by storage capability. To provide a high I/O performance in the ecosystems, Solid State Drive (SSD) are utilized as a tier or as a cache on big data distributed ecosystems. SSDs have a short lifespan that is affected by data size and the number of writing operations. Balancing performance requirements and SSD’s lifespan consumption is never easy, and it’s even harder when interacting with a huge amount of data and with heterogeneous I/O patterns. In this thesis, we analysis big data workloads I/O pattern impacts on SSD’s lifespan when SSD is used as a tier or as a cache. Then, we design a Hidden Markov Model (HMM) based I/O pattern controller that manages workload placement and guarantees cost-effective storage that enhances the workload performance by up to 60%, and improves SSD’s lifespan by up to 40%. </div><div>The designed transfer learning modeling approach and the storage management solutions improve workload modeling accuracy, and the quality of the storage management policies while the testing setup changes.<br></div>

show abstract

Storage workload modelling by hidden Markov models: Application to Flash memory

Cited by 15 publications

References 17 publications

Modeling Multi-user Behaviour in Social Networks

Modeling Multi-user Behaviour in Social Networks

OGSSim: Open Generic data Storage systems Simulation tool

Transfer Learning Based Performance Modeling And Effective Storage Management In Big Data Ecosystems

Contact Info

Product

Resources

About