We propose a modification to the rolling shutter mechanism found in CMOS detectors by shuffling the pixels in every scanline. This potential hardware modification improves the sampling of the space-time datacube, allowing the recovery of high-speed videos from a single image using either tensor completion methods or reconstruction algorithms often used for compressive temporal video. We also present a design methodology for optimal sampling schemes and compare them to random shuffling. Simulations, and experimental results obtained by optically emulating the hardware, demonstrate the ability of the shuffled rolling shutter to capture images that allow reconstructing videos, which would otherwise be impossible when using the traditional rolling shutter mechanism.

This work shows the design and training of a convolutional neural network to improve the linear response of a modulated pyramid wavefront sensor, allowing to estimate and compensate for the optical gain in real time.

The deep learning wavefront sensor (DLWFS) allows the direct estimate of Zernike coefficients of aberrated wavefronts from intensity images. The main drawback of this approach is related to the use of massive convolutional neural networks (CNNs) that are lengthy to train or estimate. In this paper, we explore several options to reduce both the training and estimation time. First, we develop a CNN that can be rapidly trained without compromising accuracy. Second, we explore the effects given smaller input image sizes and different amounts of Zernike modes to be estimated. Our simulation results demonstrate that the proposed network using images of either 8 × 8 , 16 × 16 , or 32 × 32 will dramatically reduce training time and even boost the estimation accuracy of Zernike coefficients. From our experimental results, we can confirm that a 16 × 16 DLWFS can be quickly trained and is able to estimate the first 12 Zernike coefficients–skipping piston, tip, and tilt–without sacrificing accuracy and significantly speeding up the prediction time to facilitate low-cost, real-time adaptive optics systems.

<p>Multispectral Imaging (MSI) collects a datacube of spatio-spectral information of a scene. Many acquisition methods for spectral imaging use scanning, preventing its widespread usage for dynamic scenes. On the other hand, the conventional color filter array (CFA) method often used to sample color images has also been extended to snapshot MSI using a Multispectral Filter Array (MSFA), which is a mosaic of selective spectral filters placed over the Focal Plane Array (FPA). However, even state-of-the-art MSFAs coding patterns produce artifacts and distortions in the reconstructed spectral images, which might be due to the non-optimal distribution of the spectral filters. To reduce the appearance of artifacts and provide tools for the optimal design of MSFAs, this paper proposes a novel mathematical framework to design MSFAs using a Sphere Packing (SP) approach. By assuming that each sampled filter can be represented by a sphere within the discrete datacube, SP organizes the position of the equal-size and disjoint spheres’s centers in a cubic container. Our method is denoted Multispectral Filter Array by Optimal Sphere Packing (MSFA-OSP), which seeks filter positions that maximize the minimum distance between the spheres’s centers. Simulation results show an image quality improvement of up to 2 dB and a remarkable boost in spectral similarity when using our proposed MSFA design approach for a variety of reconstruction algorithms. Moreover, MSFA-OSP notably reduces the appearance of false colors and zipper effect artifacts, often seen when using state-of-the-art demosaicking algorithms. Experiments using synthetic and real data prove that the proposed MSFA-OSP outperforms state-of-the-art MSFAs in terms of spatial and spectral fidelity.</p>

No abstract