Spatio-Temporal Attentive RNN for Node Classification in Temporal Attributed Graphs

Xu, Daolin; Cheng, Wei; Luo, Dongsheng; Liu, Xiao; Zhang, Xiang

doi:10.24963/ijcai.2019/548

Cited by 64 publications

(37 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The LSTM network is a powerful approach to learn the longterm dependency of sequential data (Xu et al 2019b;. The calculation process of the LSTM unit (applied to each time step) is described in Eqs.…”

Section: The Basic Lstm Networkmentioning

confidence: 99%

Tensorized LSTM with Adaptive Shared Memory for Learning Trends in Multivariate Time Series

Cheng²,

Zong³

et al. 2020

AAAI

Self Cite

View full text Add to dashboard Cite

The problem of learning and forecasting underlying trends in time series data arises in a variety of applications, such as traffic management, energy optimization, etc. In literature, a trend in time series is characterized by the slope and duration, and its prediction is then to forecast the two values of the subsequent trend given historical data of the time series. For this problem, existing approaches mainly deal with the case in univariate time series. However, in many real-world applications, there are multiple variables at play, and handling all of them at the same time is crucial for an accurate prediction. A natural way is to employ multi-task learning (MTL) techniques in which the trend learning of each time series is treated as a task. The key point of MTL is to learn task relatedness to achieve better parameter sharing, which however is challenging in trend prediction task. First, effectively modeling the complex temporal patterns in different tasks is hard as the temporal and spatial dimensions are entangled. Second, the relatedness among tasks may change over time. In this paper, we propose a neural network, DeepTrends, for multivariate time series trend prediction. The core module of DeepTrends is a tensorized LSTM with adaptive shared memory (TLASM). TLASM employs the tensorized LSTM to model the temporal patterns of long-term trend sequences in an MTL setting. With an adaptive shared memory, TLASM is able to learn the relatedness among tasks adaptively, based upon which it can dynamically vary degrees of parameter sharing among tasks. To further consider short-term patterns, DeepTrends utilizes a multi-task 1dCNN to learn the local time series features, and employs a task-specific sub-network to learn a mixture of long-term and short-term patterns for trend prediction. Extensive experiments on real datasets demonstrate the effectiveness of the proposed model.

show abstract

Section: The Basic Lstm Networkmentioning

confidence: 99%

Tensorized LSTM with Adaptive Shared Memory for Learning Trends in Multivariate Time Series

Cheng²,

Zong³

et al. 2020

AAAI

Self Cite

View full text Add to dashboard Cite

show abstract

“…While there have been a few attempts at applying factorization techniques (Zhou et al 2014;Stamile et al 2017;Kidzinski and Hastie 2018), and deep representation learning techniques (Xu et al 2019a;2019b), their primary focus is to improve the predictive accuracy. These techniques do not explicitly account for the complex correlation structure in the data or distinguish between random effects and fixed effects.…”

Section: Related Workmentioning

confidence: 99%

LMLFM: Longitudinal Multi-Level Factorization Machine

Liang

Sun

et al. 2020

AAAI

Self Cite

View full text Add to dashboard Cite

We consider the problem of learning predictive models from longitudinal data, consisting of irregularly repeated, sparse observations from a set of individuals over time. Such data often exhibit longitudinal correlation (LC) (correlations among observations for each individual over time), cluster correlation (CC) (correlations among individuals that have similar characteristics), or both. These correlations are often accounted for using mixed effects models that include fixed effects and random effects, where the fixed effects capture the regression parameters that are shared by all individuals, whereas random effects capture those parameters that vary across individuals. However, the current state-of-the-art methods are unable to select the most predictive fixed effects and random effects from a large number of variables, while accounting for complex correlation structure in the data and non-linear interactions among the variables. We propose Longitudinal Multi-Level Factorization Machine (LMLFM), to the best of our knowledge, the first model to address these challenges in learning predictive models from longitudinal data. We establish the convergence properties, and analyze the computational complexity, of LMLFM. We present results of experiments with both simulated and real-world longitudinal data which show that LMLFM outperforms the state-of-the-art methods in terms of predictive accuracy, variable selection ability, and scalability to data with large number of variables. The code and supplemental material is available at https://github.com/junjieliang672/LMLFM.

show abstract

“…Decoder To capture the sequential dependency in the correct words, the Gated Recurrent Unit (GRU) which has achieved great performance in many sequence learning tasks (Xu et al 2019b;Andermatt, Pezold, and Cattin 2016;Xu et al 2019a) is used as the decoder. To be specific, in the decoding process, the initial hidden state h 0 of GRU is initialized with the noised word presentationŵ.…”

Section: Learning Character-level Dependenciesmentioning

confidence: 99%

Learning Multi-Level Dependencies for Robust Word Recognition

Wang

Liu

Tang

et al. 2020

AAAI

View full text Add to dashboard Cite

Robust language processing systems are becoming increasingly important given the recent awareness of dangerous situations where brittle machine learning models can be easily broken with the presence of noises. In this paper, we introduce a robust word recognition framework that captures multi-level sequential dependencies in noised sentences. The proposed framework employs a sequence-to-sequence model over characters of each word, whose output is given to a word-level bi-directional recurrent neural network. We conduct extensive experiments to verify the effectiveness of the framework. The results show that the proposed framework outperforms state-of-the-art methods by a large margin and they also suggest that character-level dependencies can play an important role in word recognition. The code of the proposed framework and the major experiments are publicly available1.

show abstract

Spatio-Temporal Attentive RNN for Node Classification in Temporal Attributed Graphs

Cited by 64 publications

References 0 publications

Tensorized LSTM with Adaptive Shared Memory for Learning Trends in Multivariate Time Series

Tensorized LSTM with Adaptive Shared Memory for Learning Trends in Multivariate Time Series

LMLFM: Longitudinal Multi-Level Factorization Machine

Learning Multi-Level Dependencies for Robust Word Recognition

Contact Info

Product

Resources

About