Learning a Probabilistic Strategy for Computational Imaging Sensor Selection

Abstract

Optimized sensing is important for computational imaging in low-resource environments, when images must be recovered from severely limited measurements. In this paper, we propose a physics-constrained, fully differentiable, autoencoder that learns a probabilistic sensor-sampling strategy for optimized sensor design. The proposed method learns a system's preferred sampling distribution that characterizes the correlations between different sensor selections as a binary, fully-connected Ising model. The learned probabilistic model is achieved by using a Gibbs sampling inspired network architecture, and is trained end-to-end with a reconstruction network for efficient co-design. The proposed framework is applicable to sensor selection problems in a variety of computational imaging applications. In this paper, we demonstrate the approach in the context of a very-long-baseline-interferometry (VLBI) array design task, where sensor correlations and atmospheric noise present unique challenges. We demonstrate results broadly consistent with expectation, and draw attention to particular structures preferred in the telescope array geometry that can be leveraged to plan future observations and design array expansions.

End-to-End Sequential Sampling and Reconstruction for MRI

“…For example, [1,43,48] proposed co-design frameworks for 2D Cartesian κ-space sampling and [39,42] applied co-design to 2D radial κ-space sampling. 1 These methods have shown superior performance over previous baselines that combine an individually-optimized sampler and reconstructor pair [1,34,39,43,48]. However, these methods do not take advantage of the sequential nature of data collection during an MRI scan, and only solve for a generic sampling pattern for an entire dataset.…”

“…Approaches that seek to combine co-design and sequential sampling strategies have been proposed, however with only limited success thus far. The work of [14] draws inspiration from AlphaGo [31] and trains a sampler to emulate the policy distribution obtained through a Monte Carlo Tree 1 Differentiable co-design of sensing and reconstruction methods has also been successfully applied to other imaging domains as well [34]. Search (MCTS); the reconstructor is trained during alternating optimization steps.…”

“…We are instead interested in co-design, where jointly designing the two components can synergistically boost reconstruction quality. Our approach builds on neural network based co-design frameworks that have shown strong empirical performance and take advantage of efficient differentiable training [1,15,16,34].…”

Joint Optimization of Hadamard Sensing and Reconstruction in Compressed Sensing Fluorescence Microscopy

“…Deterministic sensing patterns have been jointly optimized with neural-network based reconstruction schemes in designing the color multiplexing pattern of camera sensors [4], the LED illumination pattern in Fourier ptychography [11], the optical parameters of a camera lens [19], and microscopy cell detection [28]. More recently, stochastic sampling of sensing patterns have been explored in the context of CS magnetic resonance imaging [1,32] and very-long-baseline-interferometry (VLBI) array design [22]. In this paper, we extend the idea of joint optimization of sensing strategies and neural-network based reconstruction to CS-FM.…”

“…Recently, researchers in multiple domains have explored the idea of jointly optimizing both the sensing and reconstruction parts end-to-end on training data, thus allowing for improved performance compared to separate optimization [1,22,28]. In this paper, we build on this idea and propose a method of jointly optimizing sensing and reconstruction in CS-FM.…”