Hybrid Fusion-Based Background Segmentation in Multispectral Polarimetric Imagery

Islam, Nazrul; Tahtalı, Murat; Pickering, Mark

doi:10.3390/rs12111776

Cited by 8 publications

(5 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accuracy (Acc) [47] To compare accuracy Efficiency (Eff) [47] To evaluate the performance of the models Standard Deviation (SD) [48] To estimate the contrast Average Gradient (AG) [47] To express small detail contrast and texture changes, as well as the sharpness of the image Spatial Frequency (SF) [47,48] To measure the overall activity level of the image Peak Signal-to-Noise Ratio (PSNR) [49] To compute the visual error between the fused image and the reference image Correlation Coefficient (CC) [48] To find the similarity between the reference image and the fused image Mutual Information (MI) [49] Is an indicator of how closely the two unrelated factors are related. It measures how knowledge difference between two random factors Entropy (EN) [49] Estimates uncertain index…”

Section: Metric Name Purposementioning

confidence: 99%

A Generic Framework for Enhancing Autonomous Driving Accuracy through Multimodal Data Fusion

Ignatious,

El-Sayed,

Khan

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Higher-level autonomous driving necessitates the best possible execution of important moves under all conditions. Most of the accidents in recent years caused by the AVs launched by leading automobile manufacturers are due to inadequate decision-making, which is a result of their poor perceivance of environmental information. In today’s technology-bound scenarios, versatile sensors are used by AVs to collect environmental information. Due to various technical and natural calamities, the environmental information acquired by the sensors may not be complete and clear, due to which the AVs may misinterpret the information in a different context, leading to inadequate decision-making, which may then lead to fatal accidents. To overcome this drawback, effective preprocessing of raw sensory data is a mandatory task. Pre-processing the sensory data involves two vital tasks, namely data cleaning and data fusion. Since the raw sensory data are complex and exhibit multimodal characteristics, more emphasis is given to data preprocessing. Since more innovative models have been proposed for data cleaning, this study focused on data fusion. In particular, this study proposed a generic data fusion engine, which classifies different formats of sensory data and fuses them accordingly to improve accuracy. This study proposed a generic framework to fuse the text, image, and audio data. In the first stage of this research, an innovative hybrid model was proposed to fuse multispectral image and video data. Simple and efficient models to extract the salient image features were also proposed. The hybrid image fusion model that was proposed did not yield satisfactory outcomes when combining 3D point cloud data, and its performance declined when evaluating large datasets. To address this issue, the study expanded by introducing an advanced generative adversarial network (GAN) to transform the hybrid image fusion model into a machine learning model capable of handling substantial datasets. Additionally, customized kernel functions were suggested to fuse 3D point cloud data effectively. The performance of the proposed models was assessed using standard metrics and datasets, comparing them with existing popular models. The results revealed that the proposed image fusion model outperformed the other models.

show abstract

Section: Metric Name Purposementioning

confidence: 99%

A Generic Framework for Enhancing Autonomous Driving Accuracy through Multimodal Data Fusion

Ignatious,

El-Sayed,

Khan

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Benefiting from the rich information acquired, PMI has been attracting widespread attention from researchers in various fields. Concretely, PMI shows great significance for target detection involving specular reflection inpainting 1 , background segmentation 2 and tensor representation 3 . In remote sensing, PMI is closely related to marsh vegetation classification 4 , coastal wetland classification 5 and leaf nitrogen determination 6 .…”

Section: Background and Summarymentioning

confidence: 99%

Full-Stokes polarization multispectral images of various stereoscopic objects

Fan

Teng

et al. 2023

Sci Data

View full text Add to dashboard Cite

Polarization multispectral imaging (PMI) has been applied widely with the ability of characterizing physicochemical properties of objects. However, traditional PMI relies on scanning each domain, which is time-consuming and occupies vast storage resources. Therefore, it is imperative to develop advanced PMI methods to facilitate real-time and cost-effective applications. In addition, PMI development is inseparable from preliminary simulations based on full-Stokes polarization multispectral images (FSPMI). Whereas, FSPMI measurements are always necessary due to the lack of relevant databases, which is extremely complex and severely limits PMI development. In this paper, we therefore publicize abundant FSPMI with 512 × 512 spatial pixels measured by an established system for 67 stereoscopic objects. In the system, a quarter-wave plate and a linear polarizer are rotated to modulate polarization information, while bandpass filters are switched to modulate spectral information. The required FSPMI are finally calculated from designed 5 polarization modulation and 18 spectral modulation. The publicly available FSPMI database may have the potential to greatly promote PMI development and application.

show abstract

“…Specificity (a T n fraction) is the proportion of actual negatives predicted as negatives, sensitivity (a T p fraction) the proportion of actual positives predicted as positives, G-mean the root of the product of specificity and sensitivity, and accuracy the proportion of true results obtained, either T n or T p . The mathematical evaluation measures of the aforementioned metrics are shown in Equations ( 15) to (20) [17,56].…”

Section: Selection Of Srd Metricmentioning

confidence: 99%

“…Potential applications of it could investigate acquiring an imaging system that performs image denoising [8], image dehazing [9], and semantic segmentation [10]. Multispectral imaging is a mode commonly reported in the literature for enhancing color reproduction [11], illuminant estimation [12], vegetation phenology [13,14], shadow detection [15], and background segmentation [16,17]. Additionally, although a multispectral cue is capable of generating information through penetrating deeper into an object, it is sometimes infeasible for extracting the object's inherent features.…”

Section: Introductionmentioning

confidence: 99%

Specular Reflection Detection and Inpainting in Transparent Object through MSPLFI

2021

Self Cite

View full text Add to dashboard Cite

Multispectral polarimetric light field imagery (MSPLFI) contains significant information about a transparent object’s distribution over spectra, the inherent properties of its surface and its directional movement, as well as intensity, which all together can distinguish its specular reflection. Due to multispectral polarimetric signatures being limited to an object’s properties, specular pixel detection of a transparent object is a difficult task because the object lacks its own texture. In this work, we propose a two-fold approach for determining the specular reflection detection (SRD) and the specular reflection inpainting (SRI) in a transparent object. Firstly, we capture and decode 18 different transparent objects with specularity signatures obtained using a light field (LF) camera. In addition to our image acquisition system, we place different multispectral filters from visible bands and polarimetric filters at different orientations to capture images from multisensory cues containing MSPLFI features. Then, we propose a change detection algorithm for detecting specular reflected pixels from different spectra. A Mahalanobis distance is calculated based on the mean and the covariance of both polarized and unpolarized images of an object in this connection. Secondly, an inpainting algorithm that captures pixel movements among sub-aperture images of the LF is proposed. In this regard, a distance matrix for all the four connected neighboring pixels is computed from the common pixel intensities of each color channel of both the polarized and the unpolarized images. The most correlated pixel pattern is selected for the task of inpainting for each sub-aperture image. This process is repeated for all the sub-aperture images to calculate the final SRI task. The experimental results demonstrate that the proposed two-fold approach significantly improves the accuracy of detection and the quality of inpainting. Furthermore, the proposed approach also improves the SRD metrics (with mean F1-score, G-mean, and accuracy as 0.643, 0.656, and 0.981, respectively) and SRI metrics (with mean structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), mean squared error (IMMSE), and mean absolute deviation (MAD) as 0.966, 0.735, 0.073, and 0.226, respectively) for all the sub-apertures of the 18 transparent objects in MSPLFI dataset as compared with those obtained from the methods in the literature considered in this paper. Future work will exploit the integration of machine learning for better SRD accuracy and SRI quality.

show abstract

Hybrid Fusion-Based Background Segmentation in Multispectral Polarimetric Imagery

Cited by 8 publications

References 63 publications

A Generic Framework for Enhancing Autonomous Driving Accuracy through Multimodal Data Fusion

A Generic Framework for Enhancing Autonomous Driving Accuracy through Multimodal Data Fusion

Full-Stokes polarization multispectral images of various stereoscopic objects

Specular Reflection Detection and Inpainting in Transparent Object through MSPLFI

Contact Info

Product

Resources

About