Hyperspectral image classification with retentive network

Abstract

The rapid progress in deep learning, particularly in convolutional neural networks (CNNs), has significantly enhanced the effectiveness and efficiency of hyperspectral image (HSI) classification. While CNN-based approaches excel in enriching local features, they often struggle to capture long-range dependencies in sequential data. To address this limitation, an attention mechanism can be integrated with CNN architectures to capture both global and local rich representations. Transformer architectures and their variations, known for their ability to model long-distance dependencies in sequential data, have gradually found applications in HSI classification tasks. Recently, the Retentive Network (RetNet) has emerged, claiming to offer superior scalability and efficiency compared to traditional transformers. One pivotal distinction between the self-attention operator in the Transformer and the retention mechanism in RetNet lies in the introduction of a decay parameter. This parameter explicitly regulates the attention weights assigned to each token by considering its neighboring tokens, resulting in improved performance. However, no study has been reported to show the effectiveness of RetNet for HSI analysis. In this study, we incorporate the retention mechanism and progressive neuron expansion structure into the task of pixel-wise HSI classification, and thus we name our proposed method as Retentive Progressive Expansion Network (R-PEN). Experimental analyses conducted on real-world hyperspectral image datasets have shown that the R-PEN model surpasses other pertinent deep learning models in classification performance.