Time Series Data Augmentation for Deep Learning: A Survey

Wen, Qingsong; Sun, Liang; Yang, Fan; Song, Xiaomin; Gao, Jingkun; Wang, Xue; Xu, Huan

doi:10.48550/arxiv.2002.12478

Cited by 92 publications

(113 citation statements)

References 7 publications

Supporting

Mentioning

112

Contrasting

Unclassified

Order By: Relevance

“…Take the green line as the evaluation criterion, the data point will be misclassified as an anomalous sample with a high probability. Data augmentation can reduce or even eliminate the impact of such noisy data by increasing the size and quality of the data [26,8]. In our model, we use the methods such as window slicing and STL.…”

Section: Data Augmentationmentioning

confidence: 99%

“…The current machine learning models on BGP anomaly detection are based on the dataset constructed from the records of BGP update traffic packets, which are partitioned into "anomalous" and "normal" samples. To improve the anomaly detection performance, these models mainly focus on the following two dimensions [8]: feature dimension (to choose appropriate features) and time dimension (to choose appropriate models). For traditional machine learning methods (like SVM), the data (i.e., the features) at different timestamps are considered to be independent samples [9,10,11] in which the time correlations are totally ignored.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Multi-View Framework for BGP Anomaly Detection via Graph Attention Network

Peng¹,

Nie²,

Shu³

et al. 2021

Preprint

View full text Add to dashboard Cite

As the default protocol for exchanging routing reachability information on the Internet, the abnormal behavior in traffic of Border Gateway Protocols (BGP) is closely related to Internet anomaly events. The BGP anomalous detection model ensures stable routing services on the Internet through its realtime monitoring and alerting capabilities. Previous studies either focused on the feature selection problem or the memory characteristic in data, while ignoring the relationship between features and the precise time correlation in feature (whether it's long or short term dependence). In this paper, we propose a multi-view model for capturing anomalous behaviors from BGP update traffic, in which Seasonal and Trend decomposition using Loess (STL) method is used to reduce the noise in the original time-series data, and Graph Attention Network (GAT) is used to discover feature relationships and time correlations in feature, respectively. Our results outperform the state-of-the-art methods at the anomaly detection task, with the average F1 score up to 96.3% and 93.2% on the balanced and imbalanced datasets respectively. Meanwhile, our model can be extended to classify multiple anomalous and to detect unknown events.

show abstract

Section: Data Augmentationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Multi-View Framework for BGP Anomaly Detection via Graph Attention Network

Peng¹,

Nie²,

Shu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Unlike other augmentation methods such as cropping or rotation, time-series data is mainly augmented by using specific methods like window warping [35], flipping, Fourier transform [36], and down-sampling. Among them, a simple method of adding Gaussian noise is often used [37]. This data augmentation technique improves performance in time-series prediction models such as DeepAR [38].…”

Section: Data Augmentationmentioning

confidence: 99%

A Daily Tourism Demand Prediction Framework Based on Multi-head Attention CNN: The Case of The Foreign Entrant in South Korea

Kim

Shyn

Kim

et al. 2021

Preprint

View full text Add to dashboard Cite

Developing an accurate tourism forecasting model is essential for making desirable policy decisions for tourism management. Early studies on tourism management focus on discovering external factors related to tourism demand. Recent studies utilize deep learning in demand forecasting along with these external factors. They mainly use recursive neural network models such as LSTM and RNN for their frameworks. However, these models are not suitable for use in forecasting tourism demand. This is because tourism demand is strongly affected by changes in various external factors, and recursive neural network models have limitations in handling these multivariate inputs. We propose a multi-head attention CNN model (MHAC) for addressing these limitations. The MHAC uses 1D-convolutional neural network to analyze temporal patterns and the attention mechanism to reflect correlations between input variables. This model makes it possible to extract spatiotemporal characteristics from time-series data of various variables. We apply our forecasting framework to predict inbound tourist changes in South Korea by considering external factors such as politics, disease, season, and attraction of Korean culture. The performance results of extensive experiments show that our method outperforms other deep-learning-based prediction frameworks in South Korea tourism forecasting.

show abstract

“…Further dataset details in Appendix F. ECG Data Augmentations. To augment each ECG for SimCLR (example in Appendix F, Figure 6), we apply three transformations in turn (based on prior work in time series augmentation [30,66]):…”

Section: Problem Setupmentioning

confidence: 99%

Meta-Learning to Improve Pre-Training

Raghu¹,

Lorraine²,

Kornblith³

et al. 2021

Preprint

View full text Add to dashboard Cite

Pre-training (PT) followed by fine-tuning (FT) is an effective method for training neural networks, and has led to significant performance improvements in many domains. PT can incorporate various design choices such as task and data reweighting strategies, augmentation policies, and noise models, all of which can significantly impact the quality of representations learned. The hyperparameters introduced by these strategies therefore must be tuned appropriately. However, setting the values of these hyperparameters is challenging. Most existing methods either struggle to scale to high dimensions, are too slow and memory-intensive, or cannot be directly applied to the two-stage PT and FT learning process. In this work, we propose an efficient, gradient-based algorithm to meta-learn PT hyperparameters. We formalize the PT hyperparameter optimization problem and propose a novel method to obtain PT hyperparameter gradients by combining implicit differentiation and backpropagation through unrolled optimization. We demonstrate that our method improves predictive performance on two real-world domains. First, we optimize high-dimensional task weighting hyperparameters for multitask pre-training on protein-protein interaction graphs and improve AUROC by up to 3.9%. Second, we optimize a data augmentation neural network for self-supervised PT with SimCLR on electrocardiography data and improve AUROC by up to 1.9%.The PT & FT paradigm introduces high-dimensional, complex PT hyperparameters, such as parameterized data augmentation policies used in contrastive representation learning [8,22] or the use of task, class, or instance weighting variables in multi-task PT to avoid negative transfer [70]. These hyperparameters can significantly affect the quality of pre-trained models [8], and thus finding techniques to set their values optimally is an important area of research.Choosing optimal PT hyperparameter values is challenging, and existing methods do not work well. Simple approaches such as random or grid search are inefficient since evaluating a hyperparameter setting requires performing the full, two-stage PT & FT optimization, which may be prohibitively computationally expensive. Gradient-free approaches, such as Bayesian optimization or evolutionary algorithms [33,61,47], are also limited in how well they scale to this setting. Gradient-based 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

show abstract

Time Series Data Augmentation for Deep Learning: A Survey

Cited by 92 publications

References 7 publications

A Multi-View Framework for BGP Anomaly Detection via Graph Attention Network

A Multi-View Framework for BGP Anomaly Detection via Graph Attention Network

A Daily Tourism Demand Prediction Framework Based on Multi-head Attention CNN: The Case of The Foreign Entrant in South Korea

Meta-Learning to Improve Pre-Training

Contact Info

Product

Resources

About