Multiresolution Reservoir Graph Neural Network

Pasa, Luca; Navarin, Nicolò; Sperduti, Alessandro

doi:10.1109/tnnls.2021.3090503

Cited by 23 publications

(10 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second category delves into the exploration of novel model designs and applications of RC, with an aim to enhance computational performance and efficiency in tasks related to pattern recognition. Possible solutions include (1) industrial applications such as adaptive practical nonlinear model predictive control [322] and digital twins [323]; (2) integrating RC with deep learning methods such as convolutional and graph neural networks [324,325,326,262,263]. Lastly, the third category is to keep investigating new architectures and mechanisms in physical hardware that are suitable for RC implementations.…”

Section: Physical Rc and Extremely Efficient Hardwarementioning

confidence: 99%

See 1 more Smart Citation

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Zhang

Vargas

2023

IEEE Access

View full text Add to dashboard Cite

Reservoir computing (RC), first applied to temporal signal processing, is a recurrent neural network in which neurons are randomly connected. Once initialized, the connection strengths remain unchanged. Such a simple structure turns RC into a non-linear dynamical system that maps lowdimensional inputs into a high-dimensional space. The model's rich dynamics, linear separability, and memory capacity then enable a simple linear readout to generate adequate responses for various applications. RC spans areas far beyond machine learning, since it has been shown that the complex dynamics can be realized in various physical hardware implementations and biological devices. This yields greater flexibility and shorter computation time. Moreover, the neuronal responses triggered by the model's dynamics shed light on understanding brain mechanisms that also exploit similar dynamical processes. While the literature on RC is vast and fragmented, here we conduct a unified review of RC's recent developments from machine learning to physics, biology, and neuroscience. We first review the early RC models, and then survey the state-of-the-art models and their applications. We further introduce studies on modeling the brain's mechanisms by RC. Finally, we offer new perspectives on RC development, including reservoir design, coding frameworks unification, physical RC implementations, and interaction between RC, cognitive neuroscience and evolution.

show abstract

Section: Physical Rc and Extremely Efficient Hardwarementioning

confidence: 99%

“…Last but not least, some recent works from 2019 have merged RC with other systems or paradigms such as deep learning [49,50,218,239,92,117,324,325,326,262,263]. Moreover, some authors have started investigating modifying the foundations (e.g., NG-RC [208]).…”

Section: Hybrids and New Foundationsmentioning

confidence: 99%

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Zhang

Vargas

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Dense connections use the structure of snowball connections and consistently up to more convolutional layers, and the snowball structure could concatenate multi-scale features incrementally [9]. The multi-scale information of graphs could be helpful for enhancing the expressiveness of GNNs on graph classification [26,27,28].…”

Section: Related Workmentioning

confidence: 99%

“…• DiffPool [7], GIN [20], PGCN [26], and MRGNN [28]: There are graph neural networks which have a special design for graph classification. GIN is a well-known typical GNN that was expected to achieve the ability as the Weisfeiler-Lehman graph isomorphism test, but a graph-level readout function of GIN was specially designed to produce the embedding of the entire graph for graph classification tasks [20].…”

Section: Baselinementioning

confidence: 99%

Transformer and Snowball Graph Convolution Learning for Biomedical Graph Classification

Hu¹,

Huang²,

Dong³

2023

Preprint

View full text Add to dashboard Cite

Graph or network has been widely used for describing and modeling complex systems in biomedicine. Deep learning methods, especially graph neural networks (GNNs), have been developed to learn and predict with such structured data. In this paper, we proposed a novel transformer and snowball encoding networks (TSEN) for biomedical graph classification, which introduced transformer architecture with graph snowball connection into GNNs for learning whole-graph representation. TSEN combined graph snowball connection with graph transformer by snowball encoding layers, which enhanced the power to capture multi-scale information and global patterns to learn the whole-graph features. On the other hand, TSEN also used snowball graph convolution as position embedding in transformer structure, which was a simple yet effective method for capturing local patterns naturally. Results of experiments using four graph classification datasets demonstrated that TSEN outperformed the state-of-the-art typical GNN models and the graph-transformer based GNN models.

show abstract

“…Another path that has been recently pursued to define efficient architectures is reducing the excess complexity of common DGNs by removing the non-linearities [11,12]. In addition, Reservoir Computing (RC) approaches that provide a way to implement extremely efficient alternatives to end-to-end training of Neural Networks have been extended to graph domains [13,14,15]. We also remark that message passing is not limited to neural architectures: fully unsupervised, deep and probabilistic models exist as well [16,17,18].…”

Section: The Building Blocksmentioning

confidence: 99%

Deep Learning for Graphs

Bacciu¹,

Errica²,

Navarin³

et al. 2022

ESANN 2022 Proceedings

Self Cite

View full text Add to dashboard Cite

The flourishing field of deep learning for graphs relies on the layered computation of representations from graph-structured input data. Message passing is the most common strategy for such processing of graphs, based on an efficient information exchange among the connected nodes via a local and iterative procedure. Representations learned in this way can be used to address different tasks related to nodes, edges, or even entire graphs. This tutorial paper reviews fundamental concepts and open challenges of deep learning for graphs and summarizes the contributions that have been accepted for publication to the ESANN 2022 special session on the topic.

show abstract

Multiresolution Reservoir Graph Neural Network

Cited by 23 publications

References 25 publications

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning

Transformer and Snowball Graph Convolution Learning for Biomedical Graph Classification

Deep Learning for Graphs

Contact Info

Product

Resources

About