Light Field Segmentation Using a Ray-Based Graph Structure

Superrays for Efficient Light Field Processing

“…δ A(r), A(r ) , (10) where δ is the Kronecker delta. This metric is somehow related to the Inter-Frame Label Consistency [49] for super-pixel evaluation in the case of videos, but instead of computing the consistency from frame to frame using the ground truth optical-flow, we measure the consistency between all light field views simultaneously using the ground truth depth.…”

“…However, our approach allows to coherently assign a label to all rays in the other light field views. In our assignment step we assume (as in [10]) that two light rays from the same view and close spatial coordinates are likely to image two close scene points. Therefore, a ray that is similar in appearance and close to a centroid light ray or close to one of its corresponding rays is considered likely to belong to the same scene object.…”

“…We use the dataset in [24], that is composed of 9×9 densely sampled views of 768 × 768 pixels (Papillon 2, Horses 2, Stilllife 2, Budha). We also use the dataset in [10], which is a 4 × 4 sparsely sampled light field views of 640 × 360 pixels (Scene 4). Finally, we propose a new dataset of 5 × 5 views of 640 × 480 pixels (Tricycle), which is rather sparse.…”

“…A regularisation is then run to obtain the segmentation masks using respectively depth aware and raw angular neighbourhood. To decrease the computational load of the regularisation, it is possible to merge several rays coming from the same scene point in the regularisation [10], but this relies on a relatively accurate dense depth estimation and still scales badly with the spatial light field resolution.…”

“…One objective of the paper is to enable user interaction with the whole light field while entering inputs on one view only. State of the art light field editing methods either only deal with densely sampled light fields or use a dense depth estimation to perform user-guided segmentation [8]- [10] or to propagate user inputs [11]- [15]. Despite the latest advances [16]- [19] in light field depth estimation, these methods use computationally expensive regularisation to obtain satisfactory depth maps.…”