Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis

Abstract

Recent years have witnessed the unprecedented rising of time series from almost all kindes of academic and industrial fields. Various types of deep neural network models have been introduced to time series analysis, but the important frequency information is yet lack of effective modeling. In light of this, in this paper we propose a wavelet-based neural network structure called multilevel Wavelet Decomposition Network (mWDN) for building frequency-aware deep learning models for time series analysis. mWDN preserves the advantage of multilevel discrete wavelet decomposition in frequency learning while enables the fine-tuning of all parameters under a deep neural network framework. Based on mWDN, we further propose two deep learning models called Residual Classification Flow (RCF) and multi-frequecy Long Short-Term Memory (mLSTM) for time series classification and forecasting, respectively. The two models take all or partial mWDN decomposed sub-series in different frequencies as input, and resort to the back propagation algorithm to learn all the parameters globally, which enables seamless embedding of wavelet-based frequency analysis into deep learning frameworks. Extensive experiments on 40 UCR datasets and a real-world user volume dataset demonstrate the excellent performance of our time series models based on mWDN. In particular, we propose an importance analysis method to mWDN based models, which successfully identifies those time-series elements and mWDN layers that are crucially important to time series analysis. This indeed indicates the interpretability advantage of mWDN, and can be viewed as an indepth exploration to interpretable deep learning.

Spatiotemporal traffic matrix prediction: A deep learning approach with wavelet multiscale analysis

“…In the CNN‐based TM prediction model, there are two convolution layers, where the first has 8 filters with size (3, 3) and the second has four filters with size (3, 3). Long short‐term memory, which has been used in many TM prediction tasks . In the LSTM‐based TM prediction model, there are two LSTM layers, each of which has 200 units. Wavelet decomposition‐based LSTM, which has been shown to be superior to RNN/LSTM in temporal modeling capability . In the wLSTM‐based TM prediction model, we first use Daubechies 4 wavelet to decompose original TM series into four levels TM subseries.…”

“…Kong et al applied the machine learning technologies to big data‐driven traffic flow prediction problem and constructed a novel prediction network with the LSTM model for performance measurement system. Combined with the ability of wavelet multiscale analysis, Wang et al demonstrated that wavelet decomposition‐based LSTM (wLSTM), in which wavelet decomposition was employed as a feature engineering tool before deep modeling, could be superior to traditional RNN and LSTM for time‐series forecasting tasks. However, these LSTM‐based models often ignore local spatial dependencies among multivariate time series.…”

“…Recently, neural networks and deep learning, a unified learning framework for feature extraction and data modeling, have achieved significant performance gains for TM prediction compared to traditional data‐driven models . The intuition is that the spatiotemporal feature extractions of traffic time‐series is similar to the abstraction representations of moving images or videos.…”

“…In addition, long short‐term memory (LSTM) units, a new type of recurrent neural network (RNN) specially designed for sequence modeling, have also received plenty of attentions due to their flexibility in capturing nonlinear temporal relationships. In particular, LSTM has shown its success for TM prediction in recent years . However, neither CNN nor LSTM is perfect model for TM prediction, because CNN fails to capture the temporal features while LSTM is incapable of characterizing local spatial features.…”

AlphaStock

“…A natural follow-up question is: what kind of stocks would be selected as winners by AlphaStock? To answer this question, we propose a sensitivity analysis method [1,25,26] to interpret how the history features of a stock influence its winner score in our model.…”