Scalable Time-Decaying Adaptive Prediction Algorithm

Tan, Yinyan; Zhe, Fan; Li, Guilin; Wang, Fangshan; Li, Zhengbing; Shikai, Liu; Pan, Qiuling; Xing, Eric P.; Ho, Qirong

doi:10.1145/2939672.2939714

Cited by 5 publications

(21 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering the seasonal patterns in traffic time-series datasets, four types of online seasonal adjustment factors are introduced in the OSAF+AKF algorithm. In addition, Tan et al defined a time-decaying online convex optimization problem and explored a Time-Decaying Adaptive Prediction (TDAP) algorithm for time series prediction [38]. In the biomedical field, time-series forward prediction algorithms were used for real-time brain oscillation detection and phase-locked stimulation in [8].…”

Section: Related Workmentioning

confidence: 99%

A periodicity-based parallel time series prediction algorithm in cloud computing environments

Chen

Rong

et al. 2019

Information Sciences

View full text Add to dashboard Cite

In the era of big data, practical applications in various domains continually generate large-scale time-series data. Among them, some data show significant or potential periodicity characteristics, such as meteorological and financial data. It is critical to efficiently identify the potential periodic patterns from massive time-series data and provide accurate predictions. In this paper, a Periodicity-based Parallel Time Series Prediction (PPTSP) algorithm for large-scale time-series data is proposed and implemented in the Apache Spark cloud computing environment. To effectively handle the massive historical datasets, a Time Series Data Compression and Abstraction (TSDCA) algorithm is presented, which can reduce the data scale as well as accurately extracting the characteristics. Based on this, we propose a Multi-layer Time Series Periodic Pattern Recognition (MTSPPR) algorithm using the Fourier Spectrum Analysis (FSA) method. In addition, a Periodicity-based Time Series Prediction (PTSP) algorithm is proposed. Data in the subsequent period are predicted based on all previous period models, in which a time attenuation factor is introduced to control the impact of different periods on the prediction results. Moreover, to improve the performance of the proposed algorithms, we propose a parallel solution on the Apache Spark platform, using the Streaming real-time computing module. To efficiently process the large-scale time-series datasets in distributed computing environments, Distributed Streams (DStreams) and Resilient Distributed Datasets (RDDs) are used to store and calculate these datasets. Logical and data dependencies of RDDs in the P-TSDCA, P-MTSPPR, and P-PTSP processes are considered, and the corresponding parallel execution solutions are conducted. Extensive experimental results show that our PPTSP algorithm has significant advantages compared with other algorithms in terms of prediction accuracy and performance.

show abstract

Section: Related Workmentioning

confidence: 99%

A periodicity-based parallel time series prediction algorithm in cloud computing environments

Chen

Rong

et al. 2019

Information Sciences

View full text Add to dashboard Cite

show abstract

“…in which g 1:t = is the proximal term used to ensure that the current solution does not deviate too much from past solutions with more influence given to most recent ones by using an exponential decaying function σ t,s = γ t−s with 1 > γ > 0 . This is our main difference from the original FTRL-Proximal algorithm [76] and its recently proposed variant [24]. The replacement of the per coordinate learning rate schedule by the decaying function proves to improve the prediction accuracy in the face of concept drift (discussed in Section 5.6).…”

Section: 322mentioning

confidence: 96%

“…Mirror Gradient Descent [20,21,22] Follow the Regularized Leader [23,24,25] In the first contribution, we study the periodic behavior of botnet and RAT toolkits widely found in the wild. We reply on frequency analysis to analyze the periodic patterns in the network traffic.…”

Section: Online Learningmentioning

confidence: 99%

“…On the other hand, the latter approach of FTRL is more conservative and slow in adaptation to concept drifts due to the contribution of all examples. In this aspect, TDAP [24] strikes the balance between these two approaches via the usage of a decaying factor.…”

Section: Online Machine Learningmentioning

confidence: 99%

“…In summary, we are able to extract 482 features that spans 4 different categories: API calls, registry system, file system and miscellaneous features. In this section, we additionally evaluate Support Vector Machine (SVM) [106] and two state-of-the-art FTRL algorithms: FTRL-Proximal [76] and Time-decaying Adaptive Prediction algorithms [24]. LR and SVM are two representations of batch learning paradigm while FTRL-DP, FTRL-Proximal and TDAP are three representations of online learning paradigm.…”

Section: Miscellaneous Categorymentioning

confidence: 99%

See 2 more Smart Citations