Conditional generative adversarial network demosaicing strategy for division of focal plane polarimeters

Sargent, Garrett C.; Ratliff, Bradley M.; Asari, Vijayan K.

doi:10.1364/oe.412687

Cited by 33 publications

(13 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some polarization image databases or datasets depicting polarization scenes have been reported in the literature over the last few years. [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18] They contain the capture of various scenes under natural or artificial (be it controlled or not) illumination. They present very different characteristics: some of them are partly synthetic, some of them also contain depth information, ...…”

Section: Overview Of Existing Polarization Databasesmentioning

confidence: 99%

“…In the following we will focus on the objective quality of the images. A further step would be to get interested in the polarimetric content of the images, as Sargent et al 6 did.…”

Section: Polarization Datasets Usable For Testing Demosaicing Algorithmsmentioning

confidence: 99%

“…Some polarization image databases or datasets depicting real world scenes have been reported over the past few years. [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18] Availability of high bandwidth communication networks and large data repositories allows researchers to easily share reference databases, especially with the development of open data. Their content is diverse, as well as the way they were captured.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Production of high-resolution reference polarization images from real world scenes

Bigué,

Foulonneau,

Lapray

2023

Polarization Science and Remote Sensing XI

View full text Add to dashboard Cite

A few polarization image datasets depicting real world scenes have been reported. Some of them are available on an open-data basis. Some databases contain color images, often with color bands reconstructed from a sensor equipped with a Bayer filter. Unfortunately, even if these real-world images depict a variety of objects and situations and have a good overall quality (ie the spectral bands and the various polarization channels are or can be registered, noise is reduced), they often have a low definition (smaller than 1 Mp), a low bit depth and are captured with a large lens aperture, resulting in very band-limited images. Moreover, the demosaicing procedure used to reconstruct the various color bands has a smoothing effect, reducing their resolution. This latter point proves detrimental when it comes to use these images as references for demosaicing algorithms, especially for RGB images: since each channel combining polarization direction and spectral band is very sparse in the base mosaic pattern, artifacts likely to appear are considerably underestimated with band-limited images. In this work, we review existing polarization image databases, focus on non-mosaiced datasets and propose a technique to produce HD polarization images with superior quality.

show abstract

Section: Overview Of Existing Polarization Databasesmentioning

confidence: 99%

“…In the following we will focus on the objective quality of the images. A further step would be to get interested in the polarimetric content of the images, as Sargent et al 6 did.…”

Section: Polarization Datasets Usable For Testing Demosaicing Algorithmsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Production of high-resolution reference polarization images from real world scenes

Bigué,

Foulonneau,

Lapray

2023

Polarization Science and Remote Sensing XI

View full text Add to dashboard Cite

show abstract

“…We will show in the experimental section that this approach is far from being optimal. Other recent learning-based approaches for polarimetric demosaicing are proposed in [32], [33].…”

Section: Related Workmentioning

confidence: 99%

Deep Demosaicing for Polarimetric Filter Array Cameras

Pistellato

Bergamasco

Fatima

et al. 2022

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Polarisation Filter Array (PFA) cameras allow the analysis of light polarisation state in a simple and cost-effective manner. Such filter arrays work as the Bayer pattern for colour cameras, sharing similar advantages and drawbacks. Among the others, the raw image must be demosaiced considering the local variations of the PFA and the characteristics of the imaged scene.Non-linear effects, like the cross-talk among neighbouring pixels, are difficult to explicitly model and suggest the potential advantage of a data-driven learning approach. However, the PFA cannot be removed from the sensor, making it difficult to acquire the ground-truth polarization state for training.In this work we propose a novel CNN-based model which directly demosaics the raw camera image to a per-pixel Stokes vector. Our contribution is twofold. First, we propose a network architecture composed by a sequence of Mosaiced Convolutions operating coherently with the local arrangement of the different filters. Second, we introduce a new method, employing a consumer LCD screen, to effectively acquire real-world data for training. The process is designed to be invariant by monitor gamma and external lighting conditions. We extensively compared our method against algorithmic and learning-based demosaicing techniques, obtaining a consistently lower error especially in terms of polarisation angle.

show abstract

“…Numerous image recognition based models were extended based on the CNN architecture, similar to the CGAN, and can be used in different optical applications. [23][24][25][26] The proposed CGAN consists of two networks: a generator based on U-net and a discriminator. As shown Fig.…”

Section: Proposed Deep Neural Network For Wavefront Sensingmentioning

confidence: 99%

Wavefront sensing of interference fringe based on generative adversarial network

Whang

Chen

et al. 2022

Opt Quant Electron

View full text Add to dashboard Cite

To increase the measurement accuracy of optical systems, which are implemented in various applications, an improvement of the optical measurement technique is required. This paper proposes an image-to-image wavefront sensing approach using a deep neural network that directly predicts the phase image from the corresponding interference fringe image instead of reconstruction by the Zernike coefficients. The model is based on a conditional generative adversarial network (CGAN). To train the model, we used the formulabased ideal interference fringe images as the inputs of the CGAN, to conditionally predict the corresponding phase images as the output. We numerically investigated the performance by calculating the similarity between the ideal phase image and model output. In addition, with reference to a previous study, it was determined whether the model can extract more features from the interferogram for the prediction of Zernike coefficients. Moreover, an optical simulation software was introduced to provide an increased number of actual interferograms, to verify the proposed method. Based on the results, the proposed system can obtain the phase image directly and reduce the error, thus improving the measurement accuracy of the interference fringe.

show abstract

Conditional generative adversarial network demosaicing strategy for division of focal plane polarimeters

Cited by 33 publications

References 12 publications

Production of high-resolution reference polarization images from real world scenes

Production of high-resolution reference polarization images from real world scenes

Deep Demosaicing for Polarimetric Filter Array Cameras

Wavefront sensing of interference fringe based on generative adversarial network

Contact Info

Product

Resources

About