Data Driven 2-D-to-3-D Video Conversion for Soccer

Calagari, Kiana; Elgharib, Mohamed; Didyk, Piotr; Kaspar, Alexandre; Matusik, Wojciech; Hefeeda, Mohamed

doi:10.1109/tmm.2017.2748458

Cited by 8 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, a new interesting type of depth estimation methods was introduced, which uses convolutional neural networks to support the estimation process on the basis of a previously prepared database of depth maps. Such data-driven estimation, although it can represent the direction of future research in depth estimation, is still limited to specific applications (e.g., for soccer stadiums footage [22]), stereo pairs [16] or multi-view systems with a very narrow base [17], just like conventional methods presented earlier.…”

Section: State-of-the-art-depth Estimation Methodsmentioning

confidence: 99%

Depth Map Estimation for Free-Viewpoint Television and Virtual Navigation

2020

View full text Add to dashboard Cite

The paper presents a new method of depth estimation, dedicated for free-viewpoint television (FTV) and virtual navigation (VN). In this method, multiple arbitrarily positioned input views are simultaneously used to produce depth maps characterized by high interview and temporal consistencies. The estimation is performed for segments and their size is used to control the trade-off between the quality of depth maps and the processing time of depth estimation. Additionally, an original technique is proposed for the improvement of temporal consistency of depth maps. This technique uses the temporal prediction of depth, thus depth is estimated for P-type depth frames. For such depth frames, temporal consistency is high, whereas estimation complexity is relatively low. Similarly, as for video coding, I-type depth frames with no temporal depth prediction are used in order to achieve robustness. Moreover, we propose a novel parallelization technique that significantly reduces the estimation time. The method is implemented in C++ software that is provided together with this paper, so other researchers may use it as a new reference for their future works. In performed experiments, MPEG methodology was used whenever possible. The provided results demonstrate the advantages over the Depth Estimation Reference Software (DERS) developed by MPEG. The fidelity of a depth map, measured by the quality of synthesized views, is higher on average by 2.6 dB. This significant quality improvement is obtained despite a significant reduction of the estimation time, on average 4.5 times. The application of the proposed temporal consistency enhancement method increases this reduction to 29 times. Moreover, the proposed parallelization results in the reduction of the estimation time up to 130 times (using 6 threads). As there is no commonly accepted measure of the consistency of depth maps, the application of compression efficiency of depth is proposed as a measure of depth consistency.

show abstract

Section: State-of-the-art-depth Estimation Methodsmentioning

confidence: 99%

Depth Map Estimation for Free-Viewpoint Television and Virtual Navigation

2020

View full text Add to dashboard Cite

show abstract

“…The working rule of the system is based on the structure similarity of the images. The trained (RGB+Depth) repository has been used in [29–31]. In [29], kNN (k nearest neighbors) algorithm is used to search relative images and the respective depth map images of the queried image.…”

Section: Related Workmentioning

confidence: 99%

“…Then, the corresponding depths are combined using the correlation weighting scheme. In [31], a huge synthetic (RGB + depth) database of soccer game is created. The algorithm transfers gradients of images’ depth information from synthetic dataset and find the refined depth map using spatio-temporal methodology.…”

Section: Related Workmentioning

confidence: 99%

Degraded image enhancement by image dehazing and Directional Filter Banks using Depth Image based Rendering for future free-view 3D-TV

et al. 2019

View full text Add to dashboard Cite

DIBR-3D technology has evolved over the past few years with the demands of consumers increasing in recent times for future free-view 3D videos on their home televisions. The main issue in 3D technology is the lack of 3D content available to watch using the traditional TV systems. Although, some sophisticated devices like stereoscopic cameras have been used to fill the gap between the 3D content demand and 3D content supply. But the content generated through these sophisticated devices can not be displayed on the traditional TV systems, so there needs to be some mechanism which is inline with the traditional TV. Furthermore, the huge collection of existing 2D content should be converted to 3D using depth image-based rendering techniques. This conversion technique can highly contribute in overcoming the shortage problem of the 3D content. This paper presents a novel approach for converting 2D degraded image for DIBR 3D-TV view. This degraded or noisy/blur image is enhanced through image dehazing and Directional Filter Bank (DFB). This enhancement is necessary because of the occlusion effect or hole filling problem that occurs due to imperfect depth map. The enhanced image is then segmented into the foreground image and the background image. After the segmentation, the depth map is generated using image profiles. Moreover, Stereoscopic images are finally produced using the DIBR procedure which is based on the 2D input image and the corresponding depth map. We have verified the results of the proposed approach by comparing the results with the existing state-of-the-art techniques.

show abstract

“…Among the many new forms of communication, video has the most content, and it can provide a wide range of information that is more specific, rich, and vivid than text, sound, or image [1]. Because of the variety of information contained in the video, there is a lot of video data.…”

Section: Introductionmentioning

confidence: 99%

Detection of Shot Transition in Sports Video Based on Associative Memory Neural Network

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Users must quickly and effectively classify, browse, and retrieve videos due to the explosive growth of video data. A variety of shots make up the video data stream. The most important technology in video retrieval is shot detection, which can fundamentally solve many problems, resulting in improved detection effects and even directly affecting video retrieval performance. This paper investigates the shot transition detection algorithm in digital video live broadcasts based on sporting events. To solve the problem of shot transition detection using a single training sample, an AMNN (Associative Memory Neural Network) model with online learning ability is proposed. Experiments on a large football video data set show that this algorithm detects shear and gradual change better than existing algorithms and meets the application requirements of sports video retrieval in most cases.

show abstract

Data Driven 2-D-to-3-D Video Conversion for Soccer

Cited by 8 publications

References 27 publications

Depth Map Estimation for Free-Viewpoint Television and Virtual Navigation

Depth Map Estimation for Free-Viewpoint Television and Virtual Navigation

Degraded image enhancement by image dehazing and Directional Filter Banks using Depth Image based Rendering for future free-view 3D-TV

Detection of Shot Transition in Sports Video Based on Associative Memory Neural Network

Contact Info

Product

Resources

About