Compressed sensing joint reconstruction for multi-view images

Abstract

The problem of compressed sensing joint reconstruction of multi-view images in camera networks is considered. Noting that the neighbouring images are visually similar, the multi-view correlation is captured by the sparse prior of the difference images between two contiguous multi-view images. Thus the joint reconstruction is formulated as an unconstrained optimisation problem, which contains a quadratic fidelity term and two regularisation terms encouraging the sparse priors for multi-view images and their difference images, respectively. Moreover, an effective iterative algorithm is presented to solve the optimisation problem. Experimental results with the real multi-view images show that the proposed method can perform joint reconstruction with greater accuracy than CS image-by-image reconstruction.Introduction: The multi-camera network (MCN) has received significant attention in applications such as surveillance, and robotics. The recently developed distributed compressed sensing (DCS) [1], the extension of compressed sensing (CS), can help address the challenge of acquiring higher-resolution images with incomplete sample data in the traditional MCN. The DCS decoding reconstructs all signals jointly by exploiting inter-signal correlations through the concept of joint sparsity. But these joint sparsity models are not trivial for image ensembles. Some researchers [2,3] have investigated the multi-view correlation and presented the relevant reconstruction methods. However, these methods either require estimating the relative camera positions or do not perform the reconstructions simultaneously. To date, how to reconstruct multi-view images jointly, intuitively and simultaneously via CS framework, remains a critical problem.In this Letter, we assume that the relative camera positions are unknown and the neighbouring multi-view images are visually similar. The correlation is captured by introducing a regularisation term, which encourages the sparse prior of the difference images between two contiguous multi-view images. Then the joint reconstruction is formulated as an unconstrained optimisation problem. Moreover, an effective iterative algorithm is presented to solve the optimisation problem.

View's dependency and low‐rank background‐guided compressed sensing for multi‐view image joint reconstruction

“…For example, disparity between neighbouring views captured by different cameras can be established and utilised for sparse representation. Li et al [8] assume that the gradient modulus of multi‐view images and the wavelet coefficients of the difference image are also sparse, so these prior properties could be used to build optimisation model for joint reconstruction of multi‐view images. Liu et al [9] utilise intra‐frame, inter‐view, and temporal correlations to enhance signal sparsity, and propose a novel joint reconstruction algorithm for multi‐view video reconstruction from independently CS frames and views.…”

“…According to the image decomposition model in Section 3.1, the foreground image is actually the difference image between the multi‐view image and its background. In liberate [8, 20], researchers have discussed the sparsity of this difference image, and considered that it is sparse in a certain dictionary or a set of bases, such as Haar wavelets. Here, we also use the Haar wavelet transformation, and then this sparsity can be measured by the L1 norm of Haar coefficients, …”

Correlation estimation from compressed images

“…Then, the reconstructed scene is used to refine the camera parameters and thus both the camera positions and the scene are jointly estimated using alternating minimization techniques. In another framework [22], the authors have proposed a joint reconstruction scheme based on a regularized optimization framework. The two regularization terms encourage sparse priors of multi-view images and their difference images.…”

A joint reconstruction algorithm for multi-view compressed imaging

“…In reference [4], the correlation between images is represented by local geometric transformations, and the joint reconstruction is formulated as a l 2 -l 1 optimization problem. Thirumalai and Frossard estimate the correlation of images directly from the linear measurements by using Graph Cuts algorithm to solve a regularized energy minimization problem [5] .Li et al propose to treat the joint reconstruction as an unconstrained optimization problem, which is solved by an effective iterative method [6] . Dai et al formulate the occurrences of outliers as a sparse model and solve it by a proximal-gradient algorithm [7] .…”