Energy-Efficient Spiking Segmenter for Frame and Event-Based Images

Abstract

Semantic segmentation predicts dense pixel-wise semantic labels, which is crucial for autonomous environment perception systems. For applications on mobile devices, current research focuses on energy-efficient segmenters for both frame and event-based cameras. However, there is currently no artificial neural network (ANN) that can perform efficient segmentation on both types of images. This paper introduces spiking neural network (SNN, a bionic model that is energy-efficient when implemented on neuromorphic hardware) and develops a Spiking Context Guided Network (Spiking CGNet) with substantially lower energy consumption and comparable performance for both frame and event-based images. First, this paper proposes a spiking context guided block that can extract local features and context information with spike computations. On this basis, the directly-trained SCGNet-S and SCGNet-L are established for both frame and event-based images. Our method is verified on the frame-based dataset Cityscapes and the event-based dataset DDD17. On the Cityscapes dataset, SCGNet-S achieves comparable results to ANN CGNet with 4.85 × energy efficiency. On the DDD17 dataset, Spiking CGNet outperforms other spiking segmenters by a large margin.

Smart Endoscopy Is Greener Endoscopy: Leveraging Artificial Intelligence and Blockchain Technologies to Drive Sustainability in Digestive Health Care

“…Among the initiatives that aim to reduce the carbon footprint of AI tools, one area of interest has logically focused on minimizing the energy consumption of both the algorithms employed and the associated hardware. For example, the use of Spiking Neural Networks (SNNs) has been postulated as a promising energy efficient solution to more traditional ML models [38][39][40]. Based on the thresholds established in the brain that must be overcome before a neuron will fire and action potential, SNN models are designed to only process more relevant information, thereby reducing the energy consumption associated with the use of such models without any loss of accuracy.…”

DCENet-based low-light image enhancement improved by spiking encoding and convLSTM

“…Spiking neural networks (SNNs) are frequently employed in numerous pixel-level classification tasks ( Martinez-Seras et al, 2023 ), such as object detection ( Zhang et al, 2023b ), image segmentation ( Zhang et al, 2023a ), and anomaly detection ( Yusob et al, 2018 ). Research centered on SNNs includes methods for neural network learning, data coding, and hardware platforms.…”

Advanced Methods and Applications for Neurointelligence

“…Spiking neural networks (SNNs) are frequently employed in numerous pixel-level classification tasks (Martinez-Seras et al, 2023), such as object detection (Zhang et al, 2023b), image segmentation (Zhang et al, 2023a), and anomaly detection (Yusob et al, 2018). Research centered on SNNs includes methods for neural network learning, data coding, and hardware platforms.…”