Silhouette Enhancement in Light Field Disparity Estimation Using the Structure Tensor

Lourenco, Rui; Assunção, Pedro; Tavora, L.; Fonseca-Pinto, Rui; Faria, Sérgio M. M. de

doi:10.1109/icip.2018.8451848

Cited by 6 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, it proved to be highly sensitive to texture in the background regions of the neighboring object edges, providing only a minimal penalty when the algorithm fails in regions of uniform texture. In Lourenco et al (2018), the authors of this paper proposed an edge detection based method to detect erroneous areas and inpaint them with values from the nearest background region. Despite achieving good results, the inpainting method presents some limitations as it is based only on the nearest pixels of the background region.…”

Section: Silhouette Enlargementmentioning

confidence: 99%

Enhancement of light field disparity maps by reducing the silhouette effect and plane noise

Lourenco

Assunção

Thomaz

et al. 2022

Multidim Syst Sign Process

Self Cite

View full text Add to dashboard Cite

During the last decade, there has been an increasing number of applications dealing with multidimensional visual information, either for 3D object representation or feature extraction purposes. In this context, recent advances in light field technology, have been driving research efforts in disparity estimation methods. Among the existing ones, those based on the structure tensor have emerged as very promising to estimate disparity maps from Epipolar Plane Images. However, this approach is known to have two intrinsic limitations: (i) silhouette enlargement and (ii) irregularity of surface normal maps as computed from the estimated disparity. To address these problems, this work proposes a new method for improving disparity maps obtained from the structure-tensor approach by enhancing the silhouette and reducing the noise of planar surfaces in light fields. An edge-based approach is initially used for silhouette improvement through refinement of the estimated disparity values around object edges. Then, a plane detection algorithm, based on a seed growth strategy, is used to estimate planar regions, which in turn are used to guide correction of erroneous disparity values detected in object boundaries. The proposed algorithm shows an average improvement of 98.3% in terms of median angle error for plane surfaces, when compared to regular structure-tensor-based methods, outperforming state-of-the-art methods. The proposed framework also presents very competitive results, in terms of mean square error between disparity maps and their ground truth, when compared with their counterparts.

show abstract

Section: Silhouette Enlargementmentioning

confidence: 99%

Enhancement of light field disparity maps by reducing the silhouette effect and plane noise

Lourenco

Assunção

Thomaz

et al. 2022

Multidim Syst Sign Process

Self Cite

View full text Add to dashboard Cite

show abstract

“…To reduce the computational complexity associated with match cost functions, Neri et al [ 12 ] make a local estimation based on the maximization of the total loglikelihood spatial density aggregated along the epipolar lines. Using epipolar geometry, Lourenco et al [ 13 ] first detect enlarged silhouettes, then devise a structural inpainting method to reconstruct the disparity map. Li and Jin [ 14 ] propose a novel tensor, Kullback-Leibler Divergence (KLD), to analyze the histogram distributions of the EPI’s window.…”

Section: Related Workmentioning

confidence: 99%

Depth Estimation from Light Field Geometry Using Convolutional Neural Networks

Han

Huang

Shi

et al. 2021

Sensors

View full text Add to dashboard Cite

Depth estimation based on light field imaging is a new methodology that has succeeded the traditional binocular stereo matching and depth from monocular images. Significant progress has been made in light-field depth estimation. Nevertheless, the balance between computational time and the accuracy of depth estimation is still worth exploring. The geometry in light field imaging is the basis of depth estimation, and the abundant light-field data provides convenience for applying deep learning algorithms. The Epipolar Plane Image (EPI) generated from the light-field data has a line texture containing geometric information. The slope of the line is proportional to the depth of the corresponding object. Considering the light field depth estimation as a spatial density prediction task, we design a convolutional neural network (ESTNet) to estimate the accurate depth quickly. Inspired by the strong image feature extraction ability of convolutional neural networks, especially for texture images, we propose to generate EPI synthetic images from light field data as the input of ESTNet to improve the effect of feature extraction and depth estimation. The architecture of ESTNet is characterized by three input streams, encoding-decoding structure, and skipconnections. The three input streams receive horizontal EPI synthetic image (EPIh), vertical EPI synthetic image (EPIv), and central view image (CV), respectively. EPIh and EPIv contain rich texture and depth cues, while CV provides pixel position association information. ESTNet consists of two stages: encoding and decoding. The encoding stage includes several convolution modules, and correspondingly, the decoding stage embodies some transposed convolution modules. In addition to the forward propagation of the network ESTNet, some skip-connections are added between the convolution module and the corresponding transposed convolution module to fuse the shallow local and deep semantic features. ESTNet is trained on one part of a synthetic light-field dataset and then tested on another part of the synthetic light-field dataset and real light-field dataset. Ablation experiments show that our ESTNet structure is reasonable. Experiments on the synthetic light-field dataset and real light-field dataset show that our ESTNet can balance the accuracy of depth estimation and computational time.

show abstract

“…These various applications are only possible due to the characteristics of LFs, which represent increased visual information when compared to traditional 2D images, capturing the directionality of light rays reaching the camera sensors. Such extra information enables a panoply of post-processing operations, like the extraction of depth maps [7]- [9], or image rendering with different viewing perspectives [10], or post-capture refocusing [11].…”

Section: Introductionmentioning

confidence: 99%