Epipolar Plane Diffusion: An Efficient Approach for Light Field Editing

Frigo, Oriel

doi:10.5244/c.31.47

Cited by 6 publications

(6 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our refocusing measure methodology, we first generate a focal stack using a 4D light field image. In contrast to the general focal stack with a user defined fixed step size [17,33], we define the range and the sample step size through mathematical analysis. Then, we generate a pixelwise focus map of each image in the focal stack using RA-MSFM as in (2).…”

Section: A Overall Frameworkmentioning

confidence: 99%

See 1 more Smart Citation

Refocusing Metric of Light Field Image Using Region-Adaptive Multi-Scale Focus Measure

Zhao

2022

IEEE Access

View full text Add to dashboard Cite

Compared with conventional photography, the newly emerging technique for capturing light field image has dramatically extended potential capabilities of post processing. Among the new capabilities, refocusing has received much interest. In this paper, we first investigate a region-adaptive multi-scale focus measure (RA-MSFM) which can more robustly and accurately measure focus of light field images. It is especially superior when measuring focus in flat areas where previous methods struggle. Following we design a novel refocusing metric which employs the RA-MSFM as a core technique. Using the metric, refocusing capability of a given light field image as a whole can be represented by a single number by combining focus score maps of each refocused image in the focal stack. The focus maps are generated using the proposed RA-MSFM. Quite unique from existing metrics for light field image that assess mostly image quality, our metric targets for assessing the refocusing capability. Our experiments have shown that the proposed refocusing metric not only achieves high correlation with subjective evaluations given in the form of mean opinion scores, but also produces all-in-focus images having 0.7~4.6dB higher PSNRs compared to previous state-of-the-art methods. The proposed refocusing metric can be used to measure refocusing loss in practical application such as compression, tone mapping, denoising, and smoothing, etc.INDEX TERMS Refocusing measure, light field images, multi-scale focus measure, all-in-focus, subjective experiment, refocusing capability

show abstract

Section: A Overall Frameworkmentioning

confidence: 99%

“…The main structure of this approach is illustrated in Fig. focal stack generation are not simply set as in other methods [17,33], but determined based on the rigorous mathematical analysis provided below.…”

Section: A Overall Frameworkmentioning

confidence: 99%

Refocusing Metric of Light Field Image Using Region-Adaptive Multi-Scale Focus Measure

Zhao

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…A patch-based depth-layer-aware image synthesis algorithm was adopted in [39] to propagate the edits from the centre view to all the other views. The authors in [40] use tensor driven diffusion to propagate information from the centre view along the Epipolar Plane Image (EPI) structure of the light field. These methods were used to propagate either simple edits, recolorization or inpainting from the center view to all the other views.…”

Section: Light Field Edit Propagationmentioning

confidence: 99%

A simple framework to leverage state-of-the-art single-image super-resolution methods to restore light fields

Farrugia

Guillemot²

2020

Signal Processing: Image Communication

View full text Add to dashboard Cite

Plenoptic cameras offer a cost effective solution to capture light fields by multiplexing multiple views on a single image sensor. However, the high angular resolution is achieved at the expense of reducing the spatial resolution of each view by orders of magnitude compared to the raw sensor image. While light field super-resolution is still at an early stage, the field of single image super-resolution (SISR) has recently known significant advances with the use of deep learning techniques. This paper describes a simple framework allowing us to leverage state-of-the-art SISR techniques into light fields, while taking into account specific light field geometrical constraints. The idea is to first compute a representation compacting most of the light field energy into as few components as possible. This is achieved by aligning the light field using optical flows and then by decomposing the aligned light field using singular value decomposition (SVD).The principal basis captures the information that is coherent across all the views, while the other basis contain the high angular frequencies. Super-resolving this principal basis using an SISR method allows us to super-resolve all the information that is coherent across the entire light field. In this paper, to demonstrate the interest of the approach, we have used the very deep super resolution (VDSR) method, which is one of the leading SISR algorithms, to restore the principal basis. The information restored in the principal basis is then propagated to restore all the other views using the computed optical flows. This framework allows the proposed light field super-resolution method to inherit the benefits of the SISR method used. Experimental results show that the proposed method is competitive, and most of the time superior, to recent light field super-resolution methods in terms of both PSNR and SSIM quality metrics, with a lower complexity. Moreover, the subjective results demonstrate that our method manages to restore sharper light fields which enables to generate refocused images of higher quality.

show abstract

“…where ∇ stands for the single order gradient computed using the sobel kernel. The authors in [32] compute an average weighting of the columns of T (x, s) to derive the dominant orientation, where the weights are given by an anisotropy measure. Nevertheless, the anisotropy may fail in regions that are smooth and therefore the weighted average may fail in these regions to estimate the dominant direction of the EPI.…”

Section: Light Field Reconstructionmentioning

confidence: 99%

“…In this work, we use a diffusion based inpainting algorithm that estimates the missing pixels by diffusing information available in other views. Similar to the work in [32], we exploit the EPI structure to diffuse information along the dominant orientation of the EPI. However, instead of predicting the orientation of unknown pixels from their spatial neighborhood as done in [32], we exploit the similarity between the low-and super-resolved EPIs and use the structure tensor computed on the low-resolution EPI to guide the inpainting process in the high-resolution EPI.…”

Section: Light Field Reconstructionmentioning

confidence: 99%

Light Field Super-Resolution using a Low-Rank Prior and Deep Convolutional Neural Networks

Farrugia

Guillemot

2019

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Light field imaging has recently known a regain of interest due to the availability of practical light field capturing systems that offer a wide range of applications in the field of computer vision. However, capturing high-resolution light fields remains technologically challenging since the increase in angular resolution is often accompanied by a significant reduction in spatial resolution. This paper describes a learning-based spatial light field super-resolution method that allows the restoration of the entire light field with consistency across all sub-aperture images. The algorithm first uses optical flow to align the light field and then reduces its angular dimension using low-rank approximation. We then consider the linearly independent columns of the resulting low-rank model as an embedding, which is restored using a deep convolutional neural network (DCNN). The super-resolved embedding is then used to reconstruct the remaining sub-aperture images. The original disparities are restored using inverse warping where missing pixels are approximated using a novel light field inpainting algorithm. Experimental results show that the proposed method outperforms existing light field super-resolution algorithms, achieving PSNR gains of 0.23 dB over the second best performing method. This performance can be further improved using iterative back-projection as a post-processing step.

show abstract

Epipolar Plane Diffusion: An Efficient Approach for Light Field Editing

Cited by 6 publications

References 21 publications

Refocusing Metric of Light Field Image Using Region-Adaptive Multi-Scale Focus Measure

Refocusing Metric of Light Field Image Using Region-Adaptive Multi-Scale Focus Measure

A simple framework to leverage state-of-the-art single-image super-resolution methods to restore light fields

Light Field Super-Resolution using a Low-Rank Prior and Deep Convolutional Neural Networks

Contact Info

Product

Resources

About