Adaptive sliding windows for improved estimation of data center resource utilization

Baig, Shuja-ur-Rehman; Iqbal, Waheed; Berral, Josep Lluís; Carrera, David

doi:10.1016/j.future.2019.10.026

Cited by 47 publications

(13 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…erefore, those methods train on large fixed sliding windows using an irrelevant large number of observations yielding to inaccurate estimations or fall for inaccuracy due to degradation of estimations with short windows on quick-changing trends. In this paper, we propose that the analysis for CAER is calculated based on tumbling windows on a set of updated blocks, so the system can provide up-to-date answers continuously to capture the trend for the latest resource utilization and then build an estimation model for each trend period [30].…”

Section: Methodsmentioning

confidence: 99%

An Intuitionistic Calculus to Complex Abnormal Event Recognition on Data Streams

Zhao

Guiqiu

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

Data mining in real-time data streams is associated with multiple types of uncertainty, which often leads the respective categorizers to make erroneous predictions related to the presence or absence of complex events. But recognizing complex abnormal events, even those that occur in extremely rare cases, offers significant support to decision-making systems. Therefore, there is a need for robust recognition mechanisms that will be able to predict or recognize when an abnormal event occurs or will occur on a data stream. Considering this need, this paper presents an Intuitionistic Tumbling Windows event calculus (ITWec) methodology. It is an innovative data analysis system that combines for the first time in the literature a set of multiple systems for Complex Abnormal Event Recognition (CAER). In the proposed system, the probabilities of the existence of a high-level complex abnormal event for each period are initially calculated nonparametrically, based on the probabilities of the low-level events associated with it. Because cumulative results are sought in consecutive, nonoverlapping sections of the data stream, the method uses the clearly defined rules of initialization and termination of the tumbling windows method, where there is an explicit determination of the time interval within which several blocks of a particular stream are investigated window. Finally, the number of maximum probable intervals in which an event is likely to occur based on a certain probability threshold is calculated, based on a parametric representation of intuitively fuzzy sets.

show abstract

Section: Methodsmentioning

confidence: 99%

An Intuitionistic Calculus to Complex Abnormal Event Recognition on Data Streams

Zhao

Guiqiu

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…For example, Roy et al [41] proposed an autoscaling method which uses forecasted workload and application resource utilization for the resource provisioning decisions. Baig et al [28] proposed a method for window size estimation to maximize the prediction accuracy of data center resource utilization using deep neural networks. Any regression-based estimation model can use the predicted window size method for the prediction of resource utilization with minimum error.…”

Section: Related Workmentioning

confidence: 99%

Online Workload Burst Detection for Efficient Predictive Autoscaling of Applications

et al. 2020

Self Cite

View full text Add to dashboard Cite

Autoscaling methods are employed to ensure the scalability of cloud-hosted applications. The public-facing applications are prone to receive sudden workload bursts, and the existing autoscaling methods do not handle the bursty workloads gracefully. It is challenging to detect the burst online from the incoming dynamic workload traffic, and then identifying appropriate resources to address the burst without overprovisioning is even harder. In this paper, we address this challenge by investigating the appropriate method for online burst detection and then proposed a novel predictive autoscaling method to use burst detection for satisfying specific response time requirements. We compared the proposed method with multiple state-of-the-art baseline autoscaling methods under multiple realistic and synthetic bursty workloads for a benchmark application. Our experimental results show a 60.8% average decrease in response time violations as compared to the baseline method. INDEX TERMS Autoscaling, predictive, SLO violations, response time, workload, burstiness, online burst detection.

show abstract

“…In this case, a time-dependent window-size, based on the true frequency content of the fMRI time series, could more accurately capture the time-varying FC changes in the data and is necessary to the sliding-window method. Furthermore, adaptive window-sizes have already been successfully applied in other fields, including frequent item sets mining in data stream ( Deypir et al, 2012 ; Li and Wang, 2017 ) and estimation of data center source utilization ( Baig et al, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

Single-scale time-dependent window-sizes in sliding-window dynamic functional connectivity analysis: A validation study

et al. 2020

View full text Add to dashboard Cite

During the past ten years, dynamic functional connectivity (FC) has been extensively studied using the sliding-window method. A fixed window-size is usually selected heuristically, since no consensus exists yet on choice of the optimal window-size. Furthermore, without a known ground-truth, the validity of the computed dynamic FC remains unclear and questionable. In this study, we computed single-scale time-dependent (SSTD) window-sizes for the sliding-window method. SSTD window-sizes were based on the frequency content at every time point of a time series and were computed without any prior information. Therefore, they were time-dependent and data-driven. Using simulated sinusoidal time series with frequency shifts, we demonstrated that SSTD window-sizes captured the time-dependent period (inverse of frequency) information at every time point. We further validated the dynamic FC values computed with SSTD window-sizes with both a classification analysis using fMRI data with a low sampling rate and a regression analysis using fMRI data with a high sampling rate. Specifically, we achieved both a higher classification accuracy in predicting cognitive impairment status in fighters and a larger explained behavioral variance in healthy young adults when using dynamic FC matrices computed with SSTD window-sizes as features, as compared to using dynamic FC matrices computed with the conventional fixed window-sizes. Overall, our study computed and validated SSTD window-sizes in the sliding-window method for dynamic FC analysis. Our results demonstrate that dynamic FC matrices computed with SSTD window-sizes can capture more temporal dynamic information related to behavior and cognitive function.

show abstract

Adaptive sliding windows for improved estimation of data center resource utilization

Cited by 47 publications

References 30 publications

An Intuitionistic Calculus to Complex Abnormal Event Recognition on Data Streams

An Intuitionistic Calculus to Complex Abnormal Event Recognition on Data Streams

Online Workload Burst Detection for Efficient Predictive Autoscaling of Applications

Single-scale time-dependent window-sizes in sliding-window dynamic functional connectivity analysis: A validation study

Contact Info

Product

Resources

About