No abstract

No abstract

The fusion of high-resolution multispectral (HrMSI) and low-resolution hyperspectral images (LrHSI) has been acknowledged as a promising method for generating a highresolution hyperspectral image (HrHSI), which is also termed to be an essential part for precise recognition and cataloguing of the underlying materials. In order to improve the fusion of LrHSI and HrMSI performance, in this article, we propose a novel Nonnegative Matrix Factorization Inspired Deep Unrolling Networks, dubbed NMF-DuNet, for fusing LrHSI and HrMSI. For this aim, initially, a variational fusion model regularized by non-negative sparse prior is proposed and then is solved through the gradient descent optimization method and unrolled towards the deep network. The nonnegative coefficient matrices and orthogonal of the proposed transform coefficients constraints are both incorporated into the proposed method. Moreover, the fusion of HrMSI and LrHSI heavily depends on an imaging model that explains the degeneracy of HSI in the spectral and spatial regions. Practically, the imaging model is often unknown. The degradation model is represented implicitly via a proposed network, and both the degradation model and sparse priors are jointly optimized through the training process of the proposed network. Instead of being hand-crafted, all the parameters of NMF-DuNet are learned end-to-end. Compared to the previous state-of-the-art model-based and learning-based fusion approaches, the hardware-friendly proposed NMF-DuNet outperforms both the model-based and learning-based fusion approaches and requires a far smaller number of trainable parameters and storage space while preserving real-time performance.

Hyperspectral object tracking aims to estimate the bounding box for the given target using hyperspectral data. Different from traditional color videos, hyperspectral videos have more abundant band information for their capacity to capture the reflectance spectrum of the target at a wider range of wavelengths provides important capabilities and opportunities, which provides new capabilities for discriminating targets in complex scenes, but also presents new challenges. The limited dataset and the high dimensionality of hyperspectral data are two new challenges in constructing hyperspectral trackers, resulting in existing hyperspectral tracking methods based mainly on correlation filters. This paper proposes a new Complementary Features-aware Attentive Multi-Adapter Network (CFA-MANet), which can train a neural network well and achieve high performance for Hyperspectral Object tracking just using the limited dataset. Specifically, we add a complementary features-aware module to the multi-adapter network, which employs two different strategies to reduce the dimensionality of hyperspectral data from two complementary perspectives, and the joint implementation of these two strategies results in a reduction in the amount of computed data and parameters of the designed neural network while achieving competitive results. Moreover, spatial and channel attention modules are used to learn a wider range of contexts and improve the representation of different semantic features, respectively. Crossattention is used to learn complementary information and thus generate more discriminative representations. Experimental results on hyperspectral datasets show that our method achieves the best results compared to several recent hyperspectral tracking methods.

Due to the physical boundaries, fusing low spatial resolution hyperspectral (LrHSI) with high spatial resolution multispectral (HrMSI) images is a hot and promising area for obtaining hyperspectral that have high spatial-spectral resolution images (HrHSI). Effectively formulating the fundamental features of hyperspectral images (HSI), such as global spectral correlation, nonlocal spatial correlation, as well spatial-spectral correlation, is complex in HSI-MSI fusion. Moreover, the fusion process is highly affected by the degradation systems, where these systems are not known in real scenarios. To this end, in this article, we proposed a model-guided deep unfolded fusion network with nonlocal spatial-spectral priors (MGDuNLSS-net) that can maintain the essential features of the HSIs and implicitly estimates the degradation process in an adequate running time. Specifically, the proposed method is designed based on subspace representation in an iterative manner and unrolling its steps toward a deep network as an end-to-end framework. This approach contains two submodules, fusion [nonlocal spatial-spectral block (NLSSB)] and imaging system submodules. The former submodule is proposed to exploit the images' intrinsic characteristics to improve the preservation of spectral and spatial details. NLSSB contains two nonlocal self-similarity (NLSS) layers embedded between two bidirectional simple recurrent unit (BSRU) layers. The recurrent calculation, as well as refined components to maintain the global spectral correlation, are the light recurrence operation and highway network, while 3-D convolutions in the BSRU can retain the spatial-spectral correlation. The NLSS layer can efficiently and effectively model long-range spatial contexts, which is designed based on criss-cross attention. The later submodule is used to refine the prediction of the degradation process at any iteration via backprojecting the estimated fused image to the observed pair, which can ensure the good performance of fusion. Compared with state-of-the-art fusion approaches, three remote sensing datasets are used to validate the proposed approach's performance.