Human face super-resolution on poor quality surveillance video footage

Farooq, Muhammad Umar; Dailey, Matthew N.; Mahmood, Arif; Moonrinta, Jednipat; Ekpanyapong, Mongkol

doi:10.1007/s00521-021-05973-0

Cited by 17 publications

(6 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are a few studies that learn to perform the Super-resolution (SR) task using weakly paired data. [164] uses Cycle GANs to produce super-resolved images from low-resolution ones. Both [165] and [166] attempted to leverage unregistered temporal images, albeit with different objectives; the former tackled the SR task, while the latter addressed Relative Radiometric Normalization.…”

Section: A Parallel / Strongly Paired Datamentioning

confidence: 99%

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

Kieu,

Nguyen,

Nazib

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

In remote sensing (RS), multiple modalities of data are usually available, e.g., RGB, Multispectral, Hyperspectral, LiDAR, and SAR. Multimodal machine learning systems, which fuse these rich multimodal data modalities, have shown better performance compared to unimodal systems. Most multimodal research assumes that all modalities are present, aligned, and noiseless during training and testing time. However, in real-world scenarios, it is common to observe that one or more modalities are missing, noisy, and non-aligned, in either training or testing or both. In addition, acquiring largescale, noise-free annotations is expensive, as a result, lacking sufficient annotated datasets or having to deal with inconsistent labels are open challenges. These challenges can be addressed under a learning paradigm called multimodal co-learning.This paper focuses on multimodal co-learning techniques for remote sensing data. We first review what data modalities are available in the remote sensing domain and the key benefits and challenges of combining multimodal data in the remote sensing context. We then review the remote sensing tasks that would benefit from multimodal processing including classification, segmentation, target detection, anomaly detection, and temporal change detection. We then dive deeper into technical details by reviewing more than 200 recent efforts in this area and provide a comprehensive taxonomy to systematically review state-of-the-art approaches in 4 key co-learning challenges including missing modalities, noisy modalities, limited modality annotations, and weakly-paired modalities. Based on these insights, we propose emerging research directions to inform potential future research in multimodal co-learning for remote sensing.

show abstract

Section: A Parallel / Strongly Paired Datamentioning

confidence: 99%

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

Kieu,

Nguyen,

Nazib

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…SR is an important technique used in image processing. SR is used in many fields such as medical imaging [62], satellite imaging [63][64][65], surveillance and security [66][67][68], astronomical imaging [69], military imaging [70]. It can be observed that there are some minor differences when using LR images.…”

Section: Super Resolutionmentioning

confidence: 99%

Süper Çözünürlük Yönteminin Uydu İmgelerinin Sınıflandırma Performansına Etkisi

Cengiz

Avcı

2023

Turkish Journal of Science and Technology

View full text Add to dashboard Cite

The high resolution of the image is very important for applications. Publicly available satellite images generally have low resolutions. Since low resolution causes loss of information, the desired performance cannot be achieved depending on the type of problem studied in the field of remote sensing. In such a case, super resolution algorithms are used to render low resolution images high resolution. Super resolution algorithms are used to obtain high resolution images from low resolution images. In studies with satellite images, the use of images enhanced with super resolution is important. Since the resolution of satellite images is low, the success rate in the classification process is low. In this study, super resolution method is proposed to increase the classification performance of satellite images. The attributes of satellite images were extracted using AlexNet, ResNet50, Vgg19 from deep learning architecture. Then the extracted features were then classified into 6 classes by giving input to AlexNet-Softmax, ResNet50-Softmax, Vgg19-Softmax, Support Vector Machine, K-Nearest Neighbor, decision trees and Naive Bayes classification algorithms. Without super resolution and with super resolution feature extraction and classification processes were performed separately. Classification results without super resolution and with super resolution were compared. Improvement in classification performance was observed using super resolution.

show abstract

“…Unlike image super-resolution (SR), VSR focuses more on temporal and spatial co herence between neighboring frames. SR reconstruction has applications in video display [1], vi deo surveillance [2], medical image reconstruction [3,42], and omnidirectional VSR [4]. Howe ver, SR performance is not always satisfactory, as artifacts and blocking may be introduced, le ading to temporal incoherence within video frame [5].…”

Section: Introductionmentioning

confidence: 99%

DCANet: Dual-contrast adaptive network for real-time inference video super-resolution

Lin,

Su,

Ouyang

2024

Preprint

View full text Add to dashboard Cite

Deep learning-based methods have made significant breakthroughs in real-time inference video super-resolution in recent years. However, these methods are prone to blurring, unnatural textures, and other distortion problems in super-resolution reconstructed video when dealing with complex environments and large-motion video scenes, which greatly affects the efficiency of video super-resolution reconstruction. Moreover, the real-time inference video super-resolution network training based on generative adversarial relies only on the simple modeling of features, which can obtain excellent subjective perceptual quality, but the objective index is lower has artifacts. To this end, in real-time inference video super-resolution, a dual-contrast adaptive network called DCANet is proposed to fully capture the motion offsets of neighboring frames through an adaptive optical flow network, which helps in accurate alignment. Real-time inference video super-resolution training based on generative adversarial networks relies only on simple modeling of features with poor objective metrics and artifacts. A dual-channel feature extraction module is proposed to acquire neighboring frame context features to achieve deep feature modeling. To further enhance the reconstruction quality, contrast learning is combined with the adversarial mechanism in the training strategy and a dual-contrast loss function is proposed to guide the network training. Extensive experiments on multiple benchmark testing sets of complex video scenes show that our method achieves an inference latency of 14.91ms while generating high fidelity and perceptual quality reconstructed videos, which is suitable for real-time inference in practical deployments. The code is available at https://github.com/Swaggyp1sz/DCANet.

show abstract

Human face super-resolution on poor quality surveillance video footage

Cited by 17 publications

References 31 publications

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

Multimodal Colearning Meets Remote Sensing: Taxonomy, State of the Art, and Future Works

Süper Çözünürlük Yönteminin Uydu İmgelerinin Sınıflandırma Performansına Etkisi

DCANet: Dual-contrast adaptive network for real-time inference video super-resolution

Contact Info

Product

Resources

About