Video restoration based on deep learning: a comprehensive survey

Rota, Claudio; Buzzelli, Marco; Bianco, Simone; Schettini, Raimondo

doi:10.1007/s10462-022-10302-5

Cited by 16 publications

(8 citation statements)

References 120 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Typically, servers recommend a set of videos to the user (e.g. 5), and all of these videos are pushed to the user. The user then selects which videos to watch and discards the ones they don't like, resulting in wasted transmission resources.…”

Section: Streamingmentioning

confidence: 99%

A Survey on Generative AI and LLM for Video Generation, Understanding, and Streaming

Zhou,

Wang,

Liu

et al. 2024

Preprint

View full text Add to dashboard Cite

This paper offers an insightful examination of how currently top-trending AI technologies, i.e., generative artificial intelligence (Generative AI) and large language models (LLMs), are reshaping the field of video technology, including video generation, understanding, and streaming. It highlights the innovative use of these technologies in producing highly realistic videos, a significant leap in bridging the gap between real-world dynamics and digital creation. The study also delves into the advanced capabilities of LLMs in video understanding, demonstrating their effectiveness in extracting meaningful information from visual content, thereby enhancing our interaction with videos. In the realm of video streaming, the paper discusses how LLMs contribute to more efficient and user-centric streaming experiences, adapting content delivery to individual viewer preferences. This comprehensive review navigates through the current achievements, ongoing challenges, and future possibilities of applying Generative AI and LLMs to video-related tasks, underscoring the immense potential these technologies hold for advancing the field of video technology related to multimedia, networking, and AI communities.Impact Statement-This paper contributes to the field of video technology by examining the integration of Generative AI and Large Language Models (LLMs) in video generation, understanding, and streaming. Its exploration of these technologies offers a foundational understanding of their potential and limitations in enhancing the realism and interactivity of video content. The exploration of LLMs in video comprehension sets the stage for advancements in accessibility and interaction, promising enhanced educational tools, improved user interfaces, and advanced video analytics applications. Additionally, the paper underscores the role of LLMs in optimizing video streaming services, leading to more personalized and bandwidth-efficient platforms. This could substantially benefit the entertainment sector with adaptive streaming solutions tailored to individual preferences. By identifying key challenges and future research directions, the paper guides ongoing efforts to merge AI with video technology, while raising awareness about potential ethical issues. Its influence extends beyond academia, encouraging responsible AI development and policy-making in video technology, balancing technological advancements with ethical considerations.

show abstract

Section: Streamingmentioning

confidence: 99%

A Survey on Generative AI and LLM for Video Generation, Understanding, and Streaming

Zhou,

Wang,

Liu

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, there has been a growing interest in the video (SR) problem, leading to the proposal of numerous deep learning-based models [ 2 ]. Given the need to leverage both spatial and temporal information, effectively handling the input low-resolution (LR) frames becomes crucial.…”

Section: Related Workmentioning

confidence: 99%

“…Video SR can be seen as an extension of single-image SR, which leverages spatial information along with temporal information from LR frames. It has diverse applications in video displaying [ 2 ], video surveillance [ 3 ], and satellite imagery [ 4 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Video Super-Resolution with Spatio-Temporal Modeling and Redundancy-Aware Inference

Wang,

Liu,

et al. 2023

Sensors

View full text Add to dashboard Cite

Video super-resolution aims to generate high-resolution frames from low-resolution counterparts. It can be regarded as a specialized application of image super-resolution, serving various purposes, such as video display and surveillance. This paper proposes a novel method for real-time video super-resolution. It effectively exploits spatial information by utilizing the capabilities of an image super-resolution model and leverages the temporal information inherent in videos. Specifically, the method incorporates a pre-trained image super-resolution network as its foundational framework, allowing it to leverage existing expertise for super-resolution. A fast temporal information aggregation module is presented to further aggregate temporal cues across frames. By using deformable convolution to align features of neighboring frames, this module takes advantage of inter-frame dependency. In addition, it employs a hierarchical fast spatial offset feature extraction and a channel attention-based temporal fusion. A redundancy-aware inference algorithm is developed to reduce computational redundancy by reusing intermediate features, achieving real-time inferring speed. Extensive experiments on several benchmarks demonstrate that the proposed method can reconstruct satisfactory results with strong quantitative performance and visual qualities. The real-time inferring ability makes it suitable for real-world deployment.

show abstract

“…Unlike image super-resolution (SR), VSR focuses more on temporal and spatial co herence between neighboring frames. SR reconstruction has applications in video display [1], vi deo surveillance [2], medical image reconstruction [3,42], and omnidirectional VSR [4]. Howe ver, SR performance is not always satisfactory, as artifacts and blocking may be introduced, le ading to temporal incoherence within video frame [5].…”

Section: Introductionmentioning

confidence: 99%

DCANet: Dual-contrast adaptive network for real-time inference video super-resolution

Lin,

Su,

Ouyang

2024

Preprint

View full text Add to dashboard Cite

Deep learning-based methods have made significant breakthroughs in real-time inference video super-resolution in recent years. However, these methods are prone to blurring, unnatural textures, and other distortion problems in super-resolution reconstructed video when dealing with complex environments and large-motion video scenes, which greatly affects the efficiency of video super-resolution reconstruction. Moreover, the real-time inference video super-resolution network training based on generative adversarial relies only on the simple modeling of features, which can obtain excellent subjective perceptual quality, but the objective index is lower has artifacts. To this end, in real-time inference video super-resolution, a dual-contrast adaptive network called DCANet is proposed to fully capture the motion offsets of neighboring frames through an adaptive optical flow network, which helps in accurate alignment. Real-time inference video super-resolution training based on generative adversarial networks relies only on simple modeling of features with poor objective metrics and artifacts. A dual-channel feature extraction module is proposed to acquire neighboring frame context features to achieve deep feature modeling. To further enhance the reconstruction quality, contrast learning is combined with the adversarial mechanism in the training strategy and a dual-contrast loss function is proposed to guide the network training. Extensive experiments on multiple benchmark testing sets of complex video scenes show that our method achieves an inference latency of 14.91ms while generating high fidelity and perceptual quality reconstructed videos, which is suitable for real-time inference in practical deployments. The code is available at https://github.com/Swaggyp1sz/DCANet.

show abstract

Video restoration based on deep learning: a comprehensive survey

Cited by 16 publications

References 120 publications

A Survey on Generative AI and LLM for Video Generation, Understanding, and Streaming

A Survey on Generative AI and LLM for Video Generation, Understanding, and Streaming

Real-Time Video Super-Resolution with Spatio-Temporal Modeling and Redundancy-Aware Inference

DCANet: Dual-contrast adaptive network for real-time inference video super-resolution

Contact Info

Product

Resources

About