Improving unsupervised saliency detection by migrating from RGB to multispectral images

Abstract

Saliency detection has been an important topic during the last decade. The main goal of saliency detection models is to detect the most relevant objects in a given scene. Most of these models use RGB (Red, Green, Blue) images as an input because they mainly focus on applications where features (eg, faces, textures, colors, or human silhouettes) are extracted from color images, and there are many labeled databases available for RGB‐based saliency data. Nevertheless, the use of RGB inputs clearly limits the amount of information from where to extract the salient regions as spectral information is lost during the color image recording. On the contrary, multispectral systems are able to capture more than three bands in a single capture and can retrieve information from the full spectrum at a pixel. The main aim of this study is to investigate the advantages of using multispectral images instead of RGB images for saliency detection within the framework of unsupervised models. We compare the performance of several unsupervised saliency models with both RGB and multispectral images using a specific dataset of multispectral images with ground‐truth data extracted from observers' fixation patterns. Our results show a general improvement when multispectral information is taken into account. The saliency maps estimated by using the multispectral features are closer to the ground‐truth data, with the simplest Graph‐based visual saliency and Boolean Map‐based models showing good relative gain compared with other approaches.

Eight-Channel Multispectral Image Database for Saliency Prediction

“…We also performed a comparative analysis of the heat maps obtained with the RARE model for the RGB image and the different spectral bands. Our previous study [ 26 ] showed that using the information contained in the multispectral images and adapting the saliency models, there was an improvement in the model performance. The new analysis presented in this study used only individual bands and showed that each band was not enough to adequately predict the saliency maps obtained with the RGB image.…”

“…All the metrics were computed using one of the compared heat maps as ground truth and the other as test heat map. The heat map comparison metrics studied [ 26 ] are the following: Area under the curve (AUC): three different versions of this metric were computed: AUC-Borji (AUCB) [ 16 ], AUC-Judd (AUCJ) [ 14 ], and shuffled AUC (sAUC) [ 16 ]. For different values of threshold in the heat map, true positives and false positives are computed by using the other heat map.…”

“…Recently, saliency models have been expanded to accept multispectral images as input [ 26 ] (like RGB + Near Infrared (NIR) [ 27 ] or RGB + thermal [ 28 ]), or even hyperspectral images [ 15 , 29 ]. In their recent study, İmamoğlu et al [ 29 ] proposed a CNN-based model that generates its own ground truth for hyperspectral images.…”

Conversion of Landsat 8 multispectral data through modified private content‐based image retrieval technique for secure transmission and privacy

“…Light consists of three primary colors popularly known as RGB, that is, Red-Green-Blue. 33,34 Computer screens can display images having a combination of these three colors band. When we combine these color bands, the result is a colored image having three primary color and their combination.…”

“…If we consider only a single band of the image, it will appear like a grayscale image. Light consists of three primary colors popularly known as RGB, that is, Red‐Green‐Blue 33,34 . Computer screens can display images having a combination of these three colors band.…”