Towards resource-frugal deep convolutional neural networks for hyperspectral image segmentation

Nalepa, Jakub; Antoniak, Marek; Myller, Michal; Lorenzo, Pablo Ribalta; Marcinkiewicz, M.

doi:10.1016/j.micpro.2020.102994

Cited by 55 publications

(25 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, we aim at verifying the influence of synthetic training samples on the abilities of our CNNs, especially when generated using noise-injection techniques and generative adversarial nets [100]. Finally, we work on the quantized versions of our attention-based CNNs which will be deployed on board of an imaging satellite, in a very hardware-constrained execution environment [101].…”

Section: Discussionmentioning

confidence: 99%

Hyperspectral Band Selection Using Attention-Based Convolutional Neural Networks

et al. 2020

Self Cite

View full text Add to dashboard Cite

Hyperspectral imaging has become a mature technology which brings exciting possibilities in various domains, including satellite image analysis. However, the high dimensionality and volume of such imagery is a serious problem which needs to be faced in Earth Observation applications, where efficient acquisition, transfer and storage of hyperspectral images are key factors. To reduce the time (and ultimately cost) of transferring hyperspectral data from a satellite back to Earth, various band selection algorithms have been proposed. They are built upon the observation that for a vast number of applications only a subset of all bands convey the important information about the underlying material, hence we can safely decrease the data dimensionality without deteriorating the performance of hyperspectral classification and segmentation techniques. In this paper, we introduce a novel algorithm for hyperspectral band selection that couples new attention-based convolutional neural networks used to weight the bands according to their importance with an anomaly detection technique which is exploited for selecting the most important bands. The proposed attention-based approach is data-driven, re-uses convolutional activations at different depths of a deep architecture, identifying the most informative regions of the spectrum. Also, it is modular, easy to implement, seamlessly applicable to any convolutional network, and can be trained end-to-end using gradient descent. Our rigorous experiments, performed over benchmark sets and backed up with statistical tests, showed that the deep models equipped with the attention mechanism are competitive with the stateof-the-art band selection techniques and can work orders or magnitude faster, they deliver high-quality classification, and consistently identify significant bands in the training data, permitting the creation of refined and extremely compact sets that retain the most meaningful features. Also, the attention modules do not deteriorate the classification abilities, and slow down neither training nor inference of the deep models.

show abstract

Section: Discussionmentioning

confidence: 99%

Hyperspectral Band Selection Using Attention-Based Convolutional Neural Networks

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…LIMITED-MEMORY CNN models are highly accurate, but they all have a common drawback that is they are not suitable for mobile applications or embedded systems with low power computing. In literature review, the authors in [26] introduce resource-frugal quantized convolutional neural networks to reduce their size without adversely affecting the classification capability for segmenting hyperspectral satellite images, especially focusing on the memory savings of quantized CNNs. Moreover, an approach using object class clustering to lower bit precision beyond quantization limits proposed by Prateeth Nayak, et al [27] used 3 schemes, which are uniform-ASYMM, uniform-SYMM, and power-of-2.…”

Section: Cnns Based Ssd Lite-mobile Net Methods For Object Detection Withmentioning

confidence: 99%

Applying convolutional neural networks for limited-memory application

et al. 2020

View full text Add to dashboard Cite

Currently, convolutional neural networks (CNN) are considered as the most effective tool in image diagnosis and processing techniques. In this paper, we studied and applied the modified SSDLite_MobileNetV2 and proposed a solution to always maintain the boundary of the total memory capacity in the following robust bound and applied on the bridge navigational watch & alarm system (BNWAS). The hardware was designed based on raspberry Pi-3, an embedded single board computer with CPU smartphone level, limited RAM without CUDA GPU. Experimental results showed that the deep learning model on an embedded single board computer brings us high effectiveness in application.

show abstract

“…In this paper, we thoroughly investigate the robustness of deep learning HSI segmentation algorithms against various atmospheric conditions and noise distributions that may affect the test data in the target operational environment. Specifically, we analyze spectral and spectral-spatial convolutional neural networks (CNNs) which have not only been widely applied for HSI classification [6,[15][16][17], but are also easy to be deployed in the target data processing units, exploiting e.g., field-programmable gate arrays (FPGAs) [11,18]. Since we are currently working on Intuition-1-a 6U-class satellite with a data processing unit enabling on-board data processing acquired via a hyperspectral instrument-we focus on our default acquisition targets in the atmospheric simulations, being urban and rural areas in Central Europe.…”

Section: Contributionmentioning

confidence: 99%

“…To effectively deploy machine learning HSI analysis algorithms on-board a satellite, we need to tackle not only the challenges related to the target hardware constraints, being the limited amount of available memory and computational power, but also those concerned with the acquired data [11]. The data acquisition process, and the characteristics of the captured hyperspectral imagery are dependent on various environmental and external factors, being the latitude of the satellite (alongside the target latitude), the atmospheric conditions, ground reflectance, and many more [12].…”

Section: Introductionmentioning

confidence: 99%

Towards On-Board Hyperspectral Satellite Image Segmentation: Understanding Robustness of Deep Learning through Simulating Acquisition Conditions

Nalepa

Myller²,

Cwiek³

et al. 2021

Remote Sensing

Self Cite

View full text Add to dashboard Cite

Although hyperspectral images capture very detailed information about the scanned objects, their efficient analysis, transfer, and storage are still important practical challenges due to their large volume. Classifying and segmenting such imagery are the pivotal steps in virtually all applications, hence developing new techniques for these tasks is a vital research area. Here, deep learning has established the current state of the art. However, deploying large-capacity deep models on-board an Earth observation satellite poses additional technological challenges concerned with their memory footprints, energy consumption requirements, and robustness against varying-quality image data, with the last problem being under-researched. In this paper, we tackle this issue, and propose a set of simulation scenarios that reflect a range of atmospheric conditions and noise contamination that may ultimately happen on-board an imaging satellite. We verify their impact on the generalization capabilities of spectral and spectral-spatial convolutional neural networks for hyperspectral image segmentation. Our experimental analysis, coupled with various visualizations, sheds more light on the robustness of the deep models and indicate that specific noise distributions can significantly deteriorate their performance. Additionally, we show that simulating atmospheric conditions is key to obtaining the learners that generalize well over image data acquired in different imaging settings.

show abstract

Towards resource-frugal deep convolutional neural networks for hyperspectral image segmentation

Cited by 55 publications

References 32 publications

Hyperspectral Band Selection Using Attention-Based Convolutional Neural Networks

Hyperspectral Band Selection Using Attention-Based Convolutional Neural Networks

Applying convolutional neural networks for limited-memory application

Towards On-Board Hyperspectral Satellite Image Segmentation: Understanding Robustness of Deep Learning through Simulating Acquisition Conditions

Contact Info

Product

Resources

About