Using machine learning to assess short term causal dependence and infer network links

Banerjee, Amitava; Pathak, Jaideep; Roy, Rajarshi; Restrepo, Juan G.; Ott, Edward

doi:10.1063/1.5134845

Cited by 37 publications

(18 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this context, reservoir computing (RC) [29][30][31], a version of recurrent neural network model, is effective for inference of unmeasured variables in chaotic systems using values of a known variable [32], forecasting dynamics of chaotic oscillators [33,34], predicting the evolution of the phase of chaotic dynamics [35], and prediction of critical transition in dynamical systems [36]. Also, RC is used to detect synchronization [37][38][39], spiking-bursting phenomena [40], inferring network links [41] in coupled systems. Apart from the RC, researchers have also applied different architectures of artificial neural networks such as feed-forward neural network (FFNN) [42,43], long-short term memory (LSTM) [44][45][46] for different purposes such as detecting phase transition in complex network [47], and functional connectivity in coupled systems [48], forecasting of complex dynamics [49].…”

Section: Introductionmentioning

confidence: 99%

Optimized ensemble deep learning framework for scalable forecasting of dynamics containing extreme events

Ray,

Chakraborty,

Ghosh

2021

Preprint

View full text Add to dashboard Cite

The remarkable flexibility and adaptability of both deep learning models and ensemble methods have led to the proliferation for their application in understanding many physical phenomena. Traditionally, these two techniques have largely been treated as independent methodologies in practical applications. This study develops an optimized ensemble deep learning (OEDL) framework wherein these two machine learning techniques are jointly used to achieve synergistic improvements in model accuracy, stability, scalability, and reproducibility prompting a new wave of applications in the forecasting of dynamics. Unpredictability is considered as one of the key features of chaotic dynamics, so forecasting such dynamics of nonlinear systems is a relevant issue in the scientific community. It becomes more challenging when the prediction of extreme events is the focus issue for us. In this circumstance, the proposed OEDL model based on a best convex combination of feed-forward neural networks, reservoir computing, and long short-term memory can play a key role in advancing predictions of dynamics consisting of extreme events. The combined framework can generate the best out-of-sample performance than the individual deep learners and standard ensemble framework for both numerically simulated and real world data sets. We exhibit the outstanding performance of the OEDL framework for forecasting extreme events generated from Liénard-type system, prediction of COVID-19 cases in Brazil, dengue cases in San Juan, and sea surface temperature in Niño 3.4 region.

show abstract

Section: Introductionmentioning

confidence: 99%

Optimized ensemble deep learning framework for scalable forecasting of dynamics containing extreme events

Ray,

Chakraborty,

Ghosh

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Since the seminal work by Jaeger et al [1,2], it has opened up a new direction in the field of ESN based machine learning approach [3][4][5][6][7][8]. Due to its simplicity, ESN is appeared as a considerable tool in different areas ranging from neuroscience [9,10], speech recognition [11], language processing [12], robotics [13] to stock market prediction [14], inference of connectivity [15,16], network classification [17] and even in predicting recent COVID-19 epidemic trends [18]. There are also some studies which have focused on the suitable choices of reservoir weights [4,19,20] and on the optimal range of hyper parameters [21][22][23][24][25][26][27][28][29][30], which are crucial for a good prediction.…”

Section: Introductionmentioning

confidence: 99%

Role of assortativity in predicting burst synchronization using echo state network

Roy,

Senapati,

Poria

et al. 2021

Preprint

View full text Add to dashboard Cite

“…The RC approach to machine learning has been successfully applied to a number of problems, for example, time series prediction 11 , visual identification tasks 12 , real-time detection of epileptic seizures 13 , and inferring from limited time series data; unmeasured state variables 14 , Lyapunov exponents 15 , and causal dependencies between variables 16 .…”

Section: Introductionmentioning

confidence: 99%

Multifunctionality in a Reservoir Computer

Flynn,

Tsachouridis,

Amann

2020

Preprint

View full text Add to dashboard Cite

Multifunctionality is a well observed phenomenological feature of biological neural networks and considered to be of fundamental importance to the survival of certain species over time. These multifunctional neural networks are capable of performing more than one task without changing any network connections. In this paper we investigate how this neurological idiosyncrasy can be achieved in an artificial setting with a modern machine learning paradigm known as 'Reservoir Computing'. A training technique is designed to enable a Reservoir Computer to perform tasks of a multifunctional nature. We explore the critical effects that changes in certain parameters can have on the Reservoir Computers' ability to express multifunctionality. We also expose the existence of several 'untrained attractors'; attractors which dwell within the prediction state space of the Reservoir Computer that were not part of the training. We conduct a bifurcation analysis of these untrained attractors and discuss the implications of our results.Advancements in Machine Learning often arise from a 'two-way street' between neuroscientific observation and mathematical representation. In this paper we stroll through such a street with inspiration from 'Multifunctional Neural Networks'. These are networks of neurons whose activity patterns can change on the demand of performing a given duty but synapses remain fixed. We conceptualise multifunctionality in the context of Dynamical Systems and Machine Learning by using a Reservoir Computer as a means to realise this neurological feat in an artificial setting. More specifically, we train a Reservoir Computer to imitate the dynamics of numerous chaotic attractors from different sources, based on a given initial condition. To do this we design a training technique which 'blends' and weights data from these chaotic attractors. We explore how different weightings and changes in the memory of this artificial neural network effect the desired learning outcomes. In doing so we uncover some 'behindthe-scenes' bifurcations of several other attractors found to be lurking within the prediction state space that interfere with the networks capacity to express multifunctionality. Above all, this paper identifies some new application areas suitable to a Reservoir Computer and broadens the current understanding of the dynamical capabilities inherent to this learning system.

show abstract

Using machine learning to assess short term causal dependence and infer network links

Cited by 37 publications

References 49 publications

Optimized ensemble deep learning framework for scalable forecasting of dynamics containing extreme events

Optimized ensemble deep learning framework for scalable forecasting of dynamics containing extreme events

Role of assortativity in predicting burst synchronization using echo state network

Multifunctionality in a Reservoir Computer

Contact Info

Product

Resources

About