The Time-Sequence Prediction via Temporal and Contextual Contrastive Representation Learning

Liu, Yangyang; Liu, Jian-wei

doi:10.1007/978-3-031-20862-1_34

Cited by 3 publications

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the second category consists of classification and clustering that prioritize instance-level information, i.e. coarse-grained information (Eldele et al 2021(Eldele et al , 2022Liu and wei Liu 2022), aiming to infer the target across the entire series. Therefore, when confronted with a task-agnostic pre-training model that lacks prior knowledge or awareness of specific tasks during the pre-training phase, both segment-and instance-level information become indispensable for achieving effective universal time series representation learning.…”

Section: Introductionmentioning

confidence: 99%

TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation Learning

Liu,

Chen

2024

AAAI

View full text Add to dashboard Cite

Learning universal time series representations applicable to various types of downstream tasks is challenging but valuable in real applications. Recently, researchers have attempted to leverage the success of self-supervised contrastive learning (SSCL) in Computer Vision(CV) and Natural Language Processing(NLP) to tackle time series representation. Nevertheless, due to the special temporal characteristics, relying solely on empirical guidance from other domains may be ineffective for time series and difficult to adapt to multiple downstream tasks. To this end, we review three parts involved in SSCL including 1) designing augmentation methods for positive pairs, 2) constructing (hard) negative pairs, and 3) designing SSCL loss. For 1) and 2), we find that unsuitable positive and negative pair construction may introduce inappropriate inductive biases, which neither preserve temporal properties nor provide sufficient discriminative features. For 3), just exploring segment- or instance-level semantics information is not enough for learning universal representation. To remedy the above issues, we propose a novel self-supervised framework named TimesURL. Specifically, we first introduce a frequency-temporal-based augmentation to keep the temporal property unchanged. And then, we construct double Universums as a special kind of hard negative to guide better contrastive learning. Additionally, we introduce time reconstruction as a joint optimization objective with contrastive learning to capture both segment-level and instance-level information. As a result, TimesURL can learn high-quality universal representations and achieve state-of-the-art performance in 6 different downstream tasks, including short- and long-term forecasting, imputation, classification, anomaly detection and transfer learning.

show abstract

Section: Introductionmentioning

confidence: 99%