TactileSGNet: A Spiking Graph Neural Network for Event-based Tactile Object Recognition

Sng, Weicong; Taunyazov, Tasbolat; Soh, Harold

doi:10.1109/iros45743.2020.9341421

Cited by 30 publications

(32 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, we think the location spiking neurons can build more complicated models to further boost the event-driven tactile learning performance. For example, we can develop a spiking graph neural network with location spiking neurons and combine it with [13] to better serve event-driven tactile learning tasks. Furthermore, besides event-driven tactile learning, we can apply the models with location spiking neurons to other event-driven learning fields, like event-based vision or event-driven audio sensing.…”

Section: Discussionmentioning

confidence: 99%

“…In this paper, we focus on event-driven tactile learning with SNNs. Since the development of event-driven tactile sensors is still in its infancy [13], little prior work exists on learning event-based tactile data with SNNs. The work [1] employed a neural coding scheme to convert raw tactile data from non-event-based tactile sensors into event-based spike trains.…”

Section: B Event-driven Tactile Sensing and Learningmentioning

confidence: 99%

“…• An example for the arch-like location order: [11,25,35 • An example for the whorl-like location order: [21,15,16,23,27,24,17,6,9,12,20,29,33,34,31,28,22,14,10,1,2,7,18,30,37,39,38,32,19,8,3,4,11,25,35,36,26,13,5] • An example for the loop-like location order: [1,2,3,4,5,6,7,8,9,10,11,…”

Section: ) Location Ordersmentioning

confidence: 99%

“…However, due to the high sparsity and complexity of event-driven data, learning with these sensors remains in its infancy [12]. Recently, several works [1], [11], [13] utilized Spiking Neural Networks (SNNs) [12], [14], [15] to tackle event-driven tactile learning. Unlike ANNs generally requiring expensive transformations from asynchronous discrete events to synchronous real-valued frames, SNNs can process eventbased sensor data directly.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Event - Driven Tactile Learning with Location Spiking Neurons

Kang

Banerjee

Chopp

et al. 2022

2022 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

The sense of touch is essential for a variety of daily tasks. New advances in event-based tactile sensors and Spiking Neural Networks (SNNs) spur the research in event-driven tactile learning. However, SNN-enabled event-driven tactile learning is still in its infancy due to the limited representative abilities of existing spiking neurons and high spatio-temporal complexity in the data. In this paper, to improve the representative capabilities of existing spiking neurons, we propose a novel neuron model called "location spiking neuron", which enables us to extract features of event-based data in a novel way. Moreover, based on the classical Time Spike Response Model (TSRM), we develop a specific location spiking neuron model -Location Spike Response Model (LSRM) that serves as a new building block of SNNs 1 . Furthermore, we propose a hybrid model which combines an SNN with TSRM neurons and an SNN with LSRM neurons to capture the complex spatio-temporal dependencies in the data. Extensive experiments demonstrate the significant improvements of our models over other works on event-driven tactile learning and show the superior energy efficiency of our models and location spiking neurons, which may unlock their potential on neuromorphic hardware. 2

show abstract

Section: Discussionmentioning

confidence: 99%

Section: B Event-driven Tactile Sensing and Learningmentioning

confidence: 99%

Section: ) Location Ordersmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Event - Driven Tactile Learning with Location Spiking Neurons

Kang

Banerjee

Chopp

et al. 2022

2022 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

“…Recently, there exists some works of combining SNNs with graphic scenarios, whereas these studies mainly focused on applying graph theory to analyze the features of spiking neuron and network topology [Piekniewski, 2007;Cancan, 2019;Jovanović and Rotter, 2016], or using the features of spiking neurons to solve simple graph-related problems, such as the shortest path problems, clustering problems, minimal spanning tree problems [Sala and Cios, 1999;Hamilton et al, 2020]. More recently, the work [Gu et al, 2020] introduced a graph convolution to pre-process tactile data and trained an SNN classifier. Although they achieved high performance on the classification of sensor data, this method is difficult to accommodate general graph operations and transfer to other scenarios.…”

Section: Introductionmentioning

confidence: 99%

Exploiting Spiking Dynamics with Spatial-temporal Feature Normalization in Graph Learning

Deng

et al. 2021

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Biological spiking neurons with intrinsic dynamics underlie the powerful representation and learning capabilities of the brain for processing multimodal information in complex environments. Despite recent tremendous progress in spiking neural networks (SNNs) for handling Euclidean-space tasks, it still remains challenging to exploit SNNs in processing non-Euclidean-space data represented by graph data, mainly due to the lack of effective modeling framework and useful training techniques. Here we present a general spike-based modeling framework that enables the direct training of SNNs for graph learning. Through spatial-temporal unfolding for spiking data flows of node features, we incorporate graph convolution filters into spiking dynamics and formalize a synergistic learning paradigm. Considering the unique features of spike representation and spiking dynamics, we propose a spatial-temporal feature normalization (STFN) technique suitable for SNN to accelerate convergence. We instantiate our methods into two spiking graph models, including graph convolution SNNs and graph attention SNNs, and validate their performance on three node-classification benchmarks, including Cora, Citeseer, and Pubmed. Our model can achieve comparable performance with the state-of-the-art graph neural network (GNN) models with much lower computation costs, demonstrating great benefits for the execution on neuromorphic hardware and prompting neuromorphic applications in graphical scenarios.

show abstract

Machine Learning for Tactile Perception: Advancements, Challenges, and Opportunities

Lin

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

The past decades have seen the rapid development of tactile sensors in material, fabrication, and mechanical structure design. The advancement of tactile sensors has heightened the expectation of sensor functions, and thus put forward a higher demand for data processing. However, conventional analysis techniques have not kept pace with the tactile sensor development and still suffer from some severe drawbacks, like cumbersome models, poor efficiency, and expensive costs. Machine learning, with its prominent ability for big data analysis and fast processing speed, can offer many possibilities for tactile data analysis. Herein, the machine learning techniques employed for processing tactile signals are reviewed. Supervised learning and unsupervised learning for analog signals are covered, and processing spike signals with machine learning are summarized. Furthermore, the applications in robotic tactile perception and human activity monitoring are presented. Finally, the current challenges and future prospects in sensors, data, algorithms, and benchmarks are discussed.

show abstract

TactileSGNet: A Spiking Graph Neural Network for Event-based Tactile Object Recognition

Cited by 30 publications

References 25 publications

Event - Driven Tactile Learning with Location Spiking Neurons

Event - Driven Tactile Learning with Location Spiking Neurons

Exploiting Spiking Dynamics with Spatial-temporal Feature Normalization in Graph Learning

Machine Learning for Tactile Perception: Advancements, Challenges, and Opportunities

Contact Info

Product

Resources

About