3D motion in visual saliency modeling

Wan, Pengfei; Feng, Yan; Cheung, Gene; Bajić, Ivan V.; Au, Oscar C.; Ji, Yusheng

doi:10.1109/icassp.2013.6637969

Cited by 3 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For simplicity we denote A, B for 3DMM, 3DDS [9]; and X , Y, Z for SCMM, NLDM, 3DMA respectively. Computing CM for a 3DMV channel is easy: for each sub-block in the frame, assign its conspicuity value (calculated by either (4)(5)(7)(9)(11)) to all pixels within it.…”

Section: B Feature Integrationmentioning

confidence: 99%

“…SALIENCY FROM 3D MOTION We now discuss different saliency channels derived from 3DMVs, including two old channels (3DMM and 3DDS) proposed in [9] and three new channels (SCMM, NLDM and 3DMA). They serve as additional channels to be combined with conventional channels to obtain enhanced saliency maps.…”

Section: D Motion Estimationmentioning

confidence: 99%

“…In this section, we present our 3D motion estimation algorithm (first introduced in [9]), where block matching is restricted to a 3D window consisting of a conventional 2D spatial search window and a 1D depth search window.…”

Section: D Motion Estimationmentioning

confidence: 99%

“…Specifically, this work contains three major improvements over [9]: 1) unlike 3DMM which measures motion magnitude in conventional Cartesian coordinate, SCMM measures the 3D motion magnitude in semi-spherical coordinate as suggested by psychological studies [10]; 2) unlike 3DDS with heuristic quantization of motion directions, NLDM statistically models the element of "surprise" for each detected 3D motion direction; 3) besides motion magnitude and direction, we introduce a new channel 3DMA that models motion acceleration, which has been previously shown to be a strong feature drawing human's attention [11]. In our experiments, we show that enhanced saliency maps using our new 3DMV channels correspond more closely to our collected human gaze data than conventional saliency maps [3] and those proposed in [9].…”

Section: Introductionmentioning

confidence: 99%

“…Thanks to the availability of depth frames, estimation of 3D motion vectors (3DMVs) becomes possible. In our previous work [9], 3D motion is introduced into saliency modeling for texture-plus-depth videos. We first computed 3DMV for each sub-block composed of neighboring pixels of similar depth values.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

3-D Motion Estimation for Visual Saliency Modeling

Wan

Feng

Cheung

et al. 2013

IEEE Signal Process. Lett.

Self Cite

View full text Add to dashboard Cite

Abstract-Visual saliency is a probabilistic estimate of how likely a spatial area in an image or video frame is to attract human visual attention relative to other areas. When existing bottom-up saliency models aggregate low-level features to construct a plausible saliency map, only 2D motion cues are used as motion features, even though videos typically capture dynamic 3D scenes. In this paper, we introduce 3D motion into bottom-up saliency modeling for texture-plus-depth videos. We first propose an efficient 3D motion estimation algorithm, which computes a 3D motion vector (3DMV) for each sub-block in the frame. Using the computed 3DMVs, we then derive several saliency channels (called 3DMV channels), which are incorporated into a bottom-up saliency model to obtain enhanced saliency maps. Experiments tracking human gaze show that incorporating our 3DMV channels into bottom-up saliency model significantly improves the accuracy of derived saliency maps.

show abstract

Section: B Feature Integrationmentioning

confidence: 99%

Section: D Motion Estimationmentioning

confidence: 99%

Section: D Motion Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

3-D Motion Estimation for Visual Saliency Modeling

Wan

Feng

Cheung

et al. 2013

IEEE Signal Process. Lett.

Self Cite

View full text Add to dashboard Cite

show abstract

Multiresolution Graph Fourier Transform for Compression of Piecewise Smooth Images

Cheung

Ortega

et al. 2015

IEEE Trans. on Image Process.

196

169

View full text Add to dashboard Cite

Piecewise smooth (PWS) images (e.g., depth maps or animation images) contain unique signal characteristics such as sharp object boundaries and slowly varying interior surfaces. Leveraging on recent advances in graph signal processing, in this paper, we propose to compress the PWS images using suitable graph Fourier transforms (GFTs) to minimize the total signal representation cost of each pixel block, considering both the sparsity of the signal's transform coefficients and the compactness of transform description. Unlike fixed transforms, such as the discrete cosine transform, we can adapt GFT to a particular class of pixel blocks. In particular, we select one among a defined search space of GFTs to minimize total representation cost via our proposed algorithms, leveraging on graph optimization techniques, such as spectral clustering and minimum graph cuts. Furthermore, for practical implementation of GFT, we introduce two techniques to reduce computation complexity. First, at the encoder, we low-pass filter and downsample a high-resolution (HR) pixel block to obtain a low-resolution (LR) one, so that a LR-GFT can be employed. At the decoder, upsampling and interpolation are performed adaptively along HR boundaries coded using arithmetic edge coding, so that sharp object boundaries can be well preserved. Second, instead of computing GFT from a graph in real-time via eigen-decomposition, the most popular LR-GFTs are pre-computed and stored in a table for lookup during encoding and decoding. Using depth maps and computer-graphics images as examples of the PWS images, experimental results show that our proposed multiresolution-GFT scheme outperforms H.264 intra by 6.8 dB on average in peak signal-to-noise ratio at the same bit rate.

show abstract

A Compression Technique for Piecewise Smooth Images based on Transform Coding

Varghese¹,

James²

2016

IJCA

View full text Add to dashboard Cite

Image compression has a wide range of application since it leads to reduction in storage space and easy transmission. Piecewise smooth image consists of sharp edge boundaries and smooth interior surfaces. This paper deals with compression of Piecewise smooth images using Graph Fourier Transform and Discrete Cosine Transform. In order to obtain better quality of reconstructed image blocks contains edge boundaries are transformed using DCT and smooth regions are transformed using both weighted GFT and unweighted GFT. In order to reduce the computational complexity, low pass filter and down sample a high resolution pixel block to obtain a low resolution one at the encoder, so that LR-GFT can be employed. At the decoder upsampling and interpolation are performed so that sharp edge boundaries can be preserved.

show abstract

3D motion in visual saliency modeling

Cited by 3 publications

References 17 publications

3-D Motion Estimation for Visual Saliency Modeling

3-D Motion Estimation for Visual Saliency Modeling

Multiresolution Graph Fourier Transform for Compression of Piecewise Smooth Images

A Compression Technique for Piecewise Smooth Images based on Transform Coding

Contact Info

Product

Resources

About