Benchmarking Attention-Based Interpretability of Deep Learning in Multivariate Time Series Predictions

Barić, Domjan; Fumić, Petar; Horvatić, Davor; Lipić, Tomislav

doi:10.3390/e23020143

Cited by 19 publications

(13 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another contribution by Barić et al [4], "Benchmarking Attention-Based Interpretability of Deep Learning in Multivariate Time Series Predictions," sets the newly emerging trend of specifically designing diagnostic datasets for understanding the inner workings of attention mechanism based deep learning models for multivariate forecasting tasks. The authors designed a novel benchmark of synthetically designed datasets with the transparent underlying generating process of multiple time series interactions with increasing complexity.…”

Section: Contributions To Intelligence Augmentationmentioning

confidence: 99%

Human-Centric AI: The Symbiosis of Human and Artificial Intelligence

Horvatić

Lipić

2021

Entropy

Self Cite

View full text Add to dashboard Cite

Well-evidenced advances of data-driven complex machine learning approaches emerging within the so-called second wave of artificial intelligence (AI) fostered the exploration of possible AI applications in various domains and aspects of human life, practices, and society. Most of the recent success in AI comes from the utilization of representation learning with end-to-end trained deep neural network models in tasks such as image, text,and speech recognition or strategic board and video games. By enabling automatic feature engineering, deep learning models significantly reduce the reliance on domain-expert knowledge, outperforming traditional methods based on handcrafted feature engineering and achieving performance that equals or even supersedes humans in some respects.

show abstract

Section: Contributions To Intelligence Augmentationmentioning

confidence: 99%

Human-Centric AI: The Symbiosis of Human and Artificial Intelligence

Horvatić

Lipić

2021

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…We use the implementation available in Pytorch 4 . Indeed, according to a previous study (Barić et al, 2021), it seems to be the only model with both a satisfying performance score and correct interpretability, capturing both autocorrelations and crosscorrelations between multiple time series. Interestingly, while evaluating IMV-LSTM on simulated data from statistical and mechanistic models, the correctness of interpretability increases with more complex datasets.…”

Section: Imv-lstmmentioning

confidence: 79%

Community Impact on a Cryptocurrency: Twitter Comparison Example Between Dogecoin and Litecoin

Lansiaux¹,

Tchagaspanian²,

Forget

2022

Front. Blockchain

View full text Add to dashboard Cite

Context: The third generation of cryptocurrencies gathers cryptocurrencies that are as diverse as the market is big (e.g., Dogecoin or Litecoin). While Dogecoin is seen as a memecoin, the other gathers a very different category of investors. To our knowledge, no study has independently assessed the crypto community’s economical impact on these cryptocurrencies. Furthermore, various methodological possibilities exist to forecast cryptocurrency price—mainly coming from online communities.Method: Our study has retrospectively studied (from 01/01/2015 to 03/11/2021)—using open access data—the association strength (using normalized mutual information) and the linear correlation (using Pearson’s correlation) between Twitter activity and cryptocurrency economical attributes. In addition, we have computed different models (ADF, ARIMA, and Interpretable MultiVvariable Long Short-Term Memory recurrent neural network) that forecast past price values and assessed their precision.Findings and conclusions: While the average Dogecoin transaction value is impacted by tweets, tweets are impacted by Litecoin transactions number and average Litecoin transaction value. Tweet number is impacted by Dogecoin whale behavior, but no significant relationship was found between Litecoin whales and tweets. The forecasting error resulting from our ARIMA (0,0,0) models was 0.08% (with Litecoin) and 0.22% (with Dogecoin). Therefore, those are just the beginning of scientific findings that may lead to building a trading robot based on these results. However, in itself, this study is only for academic discussion, and conclusions need to be drawn by further research. The authors cannot be liable if any financial investment is made based on its conclusions.

show abstract

“…A recent study showed that the input attention mechanism of the DA-RNN does not reflect the causal interactions between the variables composing the system [ 57 ]. This is also supported by the extremely small values of the correlation coefficient between the true interaction matrix and the input attention matrix of each gene regulatory network, reported in Table 2 .…”

Section: Resultsmentioning

confidence: 99%

Prediction of Time Series Gene Expression and Structural Analysis of Gene Regulatory Networks Using Recurrent Neural Networks

Monti

Fiorentino

Milanetti

et al. 2022

Entropy

View full text Add to dashboard Cite

Methods for time series prediction and classification of gene regulatory networks (GRNs) from gene expression data have been treated separately so far. The recent emergence of attention-based recurrent neural network (RNN) models boosted the interpretability of RNN parameters, making them appealing for the understanding of gene interactions. In this work, we generated synthetic time series gene expression data from a range of archetypal GRNs and we relied on a dual attention RNN to predict the gene temporal dynamics. We show that the prediction is extremely accurate for GRNs with different architectures. Next, we focused on the attention mechanism of the RNN and, using tools from graph theory, we found that its graph properties allow one to hierarchically distinguish different architectures of the GRN. We show that the GRN responded differently to the addition of noise in the prediction by the RNN and we related the noise response to the analysis of the attention mechanism. In conclusion, this work provides a way to understand and exploit the attention mechanism of RNNs and it paves the way to RNN-based methods for time series prediction and inference of GRNs from gene expression data.

show abstract

Benchmarking Attention-Based Interpretability of Deep Learning in Multivariate Time Series Predictions

Cited by 19 publications

References 23 publications

Human-Centric AI: The Symbiosis of Human and Artificial Intelligence

Human-Centric AI: The Symbiosis of Human and Artificial Intelligence

Community Impact on a Cryptocurrency: Twitter Comparison Example Between Dogecoin and Litecoin

Prediction of Time Series Gene Expression and Structural Analysis of Gene Regulatory Networks Using Recurrent Neural Networks

Contact Info

Product

Resources

About