Self-Supervised Time Series Representation Learning by Inter-Intra Relational Reasoning

Abstract

Self-supervised learning achieves superior performance in many domains by extracting useful representations from the unlabeled data. However, most of traditional self-supervised methods mainly focus on exploring the inter-sample structure while less efforts have been concentrated on the underlying intra-temporal structure, which is important for time series data. In this paper, we present Self-Time: a general Self-supervised Time series representation learning framework, by exploring the inter-sample relation and intra-temporal relation of time series to learn the underlying structure feature on the unlabeled time series. Specifically, we first generate the inter-sample relation by sampling positive and negative samples of a given anchor sample, and intra-temporal relation by sampling time pieces from this anchor. Then, based on the sampled relation, a shared feature extraction backbone combined with two separate relation reasoning heads are employed to quantify the relationships of the sample pairs for inter-sample relation reasoning, and the relationships of the time piece pairs for intra-temporal relation reasoning, respectively. Finally, the useful representations of time series are extracted from the backbone under the supervision of relation reasoning heads. Experimental results on multiple real-world time series datasets for time series classification task demonstrate the effectiveness of the proposed method. Code and data are publicly available at https://haoyfan.github.io/.

BIMO: Bootstrap Inter–Intra Modality at Once Unsupervised Learning for Multivariate Time Series

“…BIMO's goal is to be easily used in downstream tasks by discovering the most significant modalities for representation learning in all domains of time-series data. This study was inspired by existing work on SOTA contrastive learning-based unsupervised learning methods [15,23,27].…”

TimesNet-PM2.5: Interpretable TimesNet for Disentangling Intraperiod and Interperiod Variations in PM2.5 Prediction

“…To bridge this gap, Paparrizos et al introduced GRAIL [22], a comprehensive framework dedicated to discerning compact time-series representations that maintained the intricacies of a user-defined comparison function. Moving forward, Fan et al revealed SelfTime [23], a versatile self-supervised time-series representation of learning methodology. It honed in on the intersample and intratemporal relationships within time series to decode the latent structural features in unlabeled data.…”

CaSS: A Channel-aware Self-supervised Representation Learning Framework for Multivariate Time Series Classification

“…[12] employs the idea of contrastive learning where two positive samples are generated by weak augmentation and strong augmentation to predict each other while the similarity among different augmentations of the same sample is maximized. [13] is designed for univariate time series. It samples several segments of the time series and labels each segment pairs according to their relative distance in the origin series.…”