Semiautomatic visual-attention modeling and its application to video compression

Gitman, Yury; Ерофеев, М. В.; Vatolin, Dmitriy; Andrey, Bolshakov; Федоров, А. А.

doi:10.1109/icip.2014.7025220

Cited by 37 publications

(34 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Our experiments used the SAVAM [7] video-saliency dataset, which contains 45 videos that are each 12-18 seconds long. The original models provided by the developers of OM-CNN, ACL and SAM were trained on others datasets: SAM used SALICON [24]; ACL used SALICON, DHF1K [14], Hollywood-2 [25] and UCF sports [25]; and OM-CNN used LEDOV [13].…”

Section: Saliency Models Preparationmentioning

confidence: 99%

“…The authors of [3] showed that the performance of non-deep models can improve considerably through application of simple Table 1: Objective evaluation results of the selected models and the baselines using five metrics on the SAVAM [7] dataset. The results include the original models, the finetuned models for the training part of the SAVAM dataset (FT), and the result after applying the postprocessing transformations (PP).…”

Section: Saliency Models Preparationmentioning

confidence: 99%

See 1 more Smart Citation

Improving Video Compression with Deep Visual-attention Models

Lyudvichenko

Ерофеев

Ploshkin

et al. 2019

Proceedings of the 2019 International Conference on Intelligent Medicine and Image Processing

Self Cite

View full text Add to dashboard Cite

Recent advances in deep learning have markedly improved the quality of visual-attention modelling. In this work we apply these advances to video compression.We propose a compression method that uses a saliency model to adaptively compress frame areas in accordance with their predicted saliency. We selected three state-of-the-art saliency models, adapted them for video compression and analyzed their results. The analysis includes objective evaluation of the models as well as objective and subjective evaluation of the compressed videos.Our method, which is based on the x264 video codec, can produce videos with the same visual quality as regular x264, but it reduces the bitrate by 25% according to the objective evaluation and by 17% according to the subjective one. Also, both the subjective and objective evaluations demonstrate that saliency models can compete with gaze maps for a single observer.Our method can extend to most video bitstream formats and can improve video compression quality without requiring a switch to a new video encoding standard. video compression, saliency, visual-attention, H.264, x264 2 RELATED WORK Our proposed video-compression method aims to combine recent advances in two fields: visual-attention modelling with neural networks and saliency-aware video compression. The following discussion reviews related works from both fields. KEYWORDS

show abstract

Section: Saliency Models Preparationmentioning

confidence: 99%

Section: Saliency Models Preparationmentioning

confidence: 99%

Improving Video Compression with Deep Visual-attention Models

Lyudvichenko

Ерофеев

Ploshkin

et al. 2019

Proceedings of the 2019 International Conference on Intelligent Medicine and Image Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Examples of databases include: VQEG FRTV Phase I and HDTV that contains SD and HD television sequences [46], [47]; LIVE database analysed MPEG-2 and H.264 compression, and simulated transmissions over IP wired and wireless networks [48]; LIVE Mobile database contains H.264 compressed videos with distortions such as packet loss, frame freeze, and rate adaptation, that were evaluated on smartphone and tablet mobile devices [49], [50]; ECVQ and EVVQ databases contain MPEG-4 and H.264 compressed videos at CIF and VGA resolution [51], [52]; MMSP SVD database contains sequences encoded using scalable video coding [53]; SAVAM database contains eye-tracking data for video sequences at HD and UHD resolution [54].…”

Section: Public Vqa Databasesmentioning

confidence: 99%

VQAMap: A Novel Mechanism for Mapping Objective Video Quality Metrics to Subjective MOS Scale

Moldovan

Ghergulescu²,

Muntean

2016

IEEE Trans. on Broadcast.

View full text Add to dashboard Cite

The reliable estimation of video quality has become increasingly important with the proliferation of online video services and users becoming more quality aware. A multitude of objective video quality assessment (VQA) metrics with various performance and complexity have been proposed. However, their applicability in real-world scenarios is limited by the lack of clear interpretations of how the metric values reflect the subjective user-perceived video quality. This paper proposes a novel mechanism called VQAMap, that uses data from public VQA databases and enables to automatically create generic rules for mapping the values of objective VQA metrics to the subjective MOS scale (i.e., 1 -bad, 2 -poor, 3 -fair, 4 -good, and 5 -excellent). An extensive evaluation study of VQAMap was conducted using data from three public VQA databases, considering six objective VQA metrics. The results analysis has shown that VQAMap provides mapping rules with quality estimation accuracy as high as 95%, while the variation in performance being caused by the varying accuracy of the different objective metrics.Index Terms-Video quality assessment (VQA), mean opinion score (MOS), subjective methods, objective metrics, video quality mapping, performance evaluation, public video quality databases.

show abstract

“…(a) (b) saliency 맵을 쉽게 구하는 방법이 소개되고 있다 [5]. 또한 crowd sourcing을 사용하여 적은 비용으로 짧 은 시간에 saliency 맵을 구하는 연구도 새롭게 진행 되고 있다 [6].…”

unclassified

Saliency Map Based Color Image Compression for Visual Quality Enhancement of Image

Jung¹

2017

Journal of Korea Multimedia Society

View full text Add to dashboard Cite

A color image compression based on saliency map was proposed. The proposed method provides higher quality in saliency blocks on which people's attention focuses, compared with non-saliency blocks on which the attention less focuses at a given bitrate. The proposed method uses 3 different quantization tables according to each block's saliency level. In the experiment using 6 typical images, we compared the proposed method with JPEG and other conventional methods. As the result, it showed that the proposed method (Qup=0.5*Qx) is about 3.1 to 1.2 dB better than JPEG and others in saliency blocks in PSNR at the almost similar bitrate. In the comparison of result images, the proposed one also showed less error than others in saliency blocks.

show abstract

Semiautomatic visual-attention modeling and its application to video compression

Cited by 37 publications

References 22 publications

Improving Video Compression with Deep Visual-attention Models

Improving Video Compression with Deep Visual-attention Models

VQAMap: A Novel Mechanism for Mapping Objective Video Quality Metrics to Subjective MOS Scale

Saliency Map Based Color Image Compression for Visual Quality Enhancement of Image

Contact Info

Product

Resources

About