Edge-based depth gradient refinement for 2D to 3D learned prior conversion

Herrera, Juan López de; del-Blanco, Carlos R.; García, Narciso

doi:10.1109/3dtv.2015.7169364

Cited by 6 publications

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To alleviate this situation, we proposed in [7] an algorithm that performed a hierarchical search in a previously clustered database., improving significantly the efficiency of the search process. In [8] we used GIST as a descriptor since it improves the performance of the algorithm, and introduced edge-based refinement post-processing to enhance the 3D structure of the scene.…”

Section: Introductionmentioning

confidence: 99%

Improved 2D-to-3D video conversion by fusing optical flow analysis and scene depth learning

Herrera

del-Blanco

García

2016

2016 3dtv-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3dtv-Con)

Self Cite

View full text Add to dashboard Cite

Automatic 2D-to-3D conversion aims to reduce the existing gap between the scarce 3D content and the incremental amount of displays that can reproduce this 3D content. Here, we present an automatic 2D-to-3D conversion algorithm that extends the functionality of the most of the existing machine learning based conversion approaches to deal with moving objects in the scene, and not only with static backgrounds. Under the assumption that images with a high similarity in color have likely a similar 3D structure, the depth of a query video sequence is inferred from a color + depth training database. First, a depth estimation for the background of each image of the query video is computed adaptively by combining the depths of the most similar images to the query ones. Then, the use of optical flow enhances the depth estimation of the different moving objects in the foreground. Promising results have been obtained in a public and widely used database.

show abstract

Section: Introductionmentioning

confidence: 99%

Improved 2D-to-3D video conversion by fusing optical flow analysis and scene depth learning

Herrera

del-Blanco

García

2016

2016 3dtv-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3dtv-Con)

Self Cite

View full text Add to dashboard Cite

show abstract

“…When the algorithm cannot find images that are similar enough, this step is skipped to avoid to deteriorate the conversion. This contribution was published in [12].…”

Section: Depth Estimationmentioning

confidence: 99%

“…Edge enhancement improvement. An adaptive size filtering stage to refine the depth estimations was first introduced in [12]. A new segmentation-based version of the filtering task, which outperforms the previous results was published in [8].…”

Section: Contributionsmentioning

confidence: 99%

“…In the second branch, an optical flow analysis of the scene is used to detect and extract the moving objects in the foreground of the scene. Finally, the background and the foreground are combined to compute The main differences with the approach presented in Chapter 4 ( [2,11,12]) are the extension of the method from only images to video sequences using optical flow to detect moving objects in the scene, the independent computation of background and foreground, instead of obtaining the global depth in one step, and the computation of just one background for the whole video sequence, instead of doing it for every frame as others methods do, thus improving the computational efficiency of the algorithm.…”

Section: Systemmentioning

confidence: 99%

See 1 more Smart Citation

2D to 3D image and video conversion using machine learning

Conejero¹

View full text Add to dashboard Cite

Despite the concept of three-dimensional (3D) image and video was introduced many years ago, there has been recently a significant increase in the number of available 3D displays and players in the last years. Nevertheless, the amount of 3D content has not increased in the same magnitude, creating a gap between 3D offer and demand. To reduce this difference, many algorithms have appeared that perform 2D-to-3D image and video conversion. While many of this techniques usually required several images from the same scene to perform the conversion, the most recent family of these techniques, which corresponds to machine learningbased algorithms, is not restricted by this limitation computing the 3D image with a single view of the scene. Machine learning-based methods require the use of databases of 2D and 3D images to learn how to perform this conversion. Since the number of available datasets has recently increased significantly, these algorithms have become very popular. However, the quality achieved by the current 2D to 3D conversion techniques is far to be fully satisfying and they need to be improved in order be used for producing good quality 3D content. This thesis proposes two systems for 2D-to-3D conversion, one for image and another one for video, belonging to the machine learning family of methods. With respect to the image conversion system, a new approach is proposed that makes the algorithm more robust and adaptive to different types of scenarios by using a combination of feature descriptors. At the same time, the proposed clustering of the dataset makes the solution faster and more efficient to deal with large datasets. The system also learns how to automatically adapt the value of the different parameters involved in the conversion, resulting in a fully automatic solution.With respect to video conversion, the learning based approach for images is extended to video sequences. The algorithm is divided in three main parts. In the first one, a depth estimation of the background is computed using the previous presented approach. Then, the foreground is segmented analyzing the optical flow to manage the different objects individually.Finally, the depth estimation of the background is combined with the foreground information and filtered to obtain the final depth estimation.The developed algorithms have been tested with different publicly available datasets of 3D images and video sequences.

show abstract

“…• 2D to 3D conversion: This method produces the depth information from an existent 2D video using different algorithms [34][35][36][37][38][39][40][41][42][43][44]. As an example of such algorithms, Philips announced the WOWvx BlueBox that provides a 3D clip from a 2D video in a semiautomatic algorithm [44].…”

Section: 1 3d /M U L T I V I Ew V I D E O R E P R E S E N T a T I O Nmentioning

confidence: 99%

Reliable Streaming of Stereoscopic Video Considering Region of Interest

Rahimi¹

View full text Add to dashboard Cite

3D video applications are growing increasingly common as the required infrastructure and technology to stream 3D video becomes more predominant. However, the quality of displayed videos may fluctuate due to packet failure as an integral part of either wired or wireless streaming networks. Therefore, more robust methods of video streaming have always been fascinating to show more favourable efficiency outcomes.This thesis first examines different video streaming techniques and compares the pros and cons of each technique. It then introduces a new streaming method that applies to 3D video for live video streaming applications especially for a sporting event or other live video applications.To this end, the thesis describes how a 3D video is captured and represented, and how humans perceive the 3D scene. Considering the pros and cons of current video streaming techniques and intended applications, the proposed method introduces a new multiple description coding (MDC) method focusing on interesting objects of the scene, called the region of interest (ROI). It is worth mentioning that a new technique, using the scene's depth information, is used to extract the ROI. This technique is not as complex as learning algorithms are, and there is no need to train the algorithm. Since the human eye is more sensitive to objects than pixels, this method can also provide better performance from the point of subjective assessment (which is out of focus of this thesis) because the proposed method focuses on important objects of the scene and assigns more bandwidth to them. Heartfelt gratitude and love go to my parents, Zahra and Mohammadali, for providing me with abundant support and continuous encouragement throughout my years of study.Last, but not least, I wish to express my infinite appreciation and love to my wife, Elham. Without her endless patience and unfailing support, completing this thesis would not have been possible.

show abstract

Edge-based depth gradient refinement for 2D to 3D learned prior conversion

Cited by 6 publications

References 12 publications

Improved 2D-to-3D video conversion by fusing optical flow analysis and scene depth learning

Improved 2D-to-3D video conversion by fusing optical flow analysis and scene depth learning

2D to 3D image and video conversion using machine learning

Reliable Streaming of Stereoscopic Video Considering Region of Interest

Contact Info

Product

Resources

About