Using eye-tracking to assess different image retargeting methods

Castillo, Susana; Judd, Tilke; Gutiérrez, Diego

doi:10.1145/2077451.2077453

Cited by 46 publications

(20 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors observed that: 1) even strong artifacts in the retargeted photo cannot influence human gaze shifting if they are distributed outside the regions of interest; 2) removing contents in photo retargeting might change its semantics, which influences human perception of photo esthetics accordingly; and 3) employing eye-tracking data can more accurately reflect the regions of interest, which might be helpful for photo retargeting. Consistent with [33], our experiments also show that human gaze shifting paths can substantially increase retargeting performance. Since it is impractical to use human data for real-world retargeting tasks, our method aims to generate AGPs to mimick human gaze shifting, where experimental results show a high degree of consistency (over 90%) of our AGP and real human gaze shifting path.…”

Section: Hsupporting

confidence: 84%

“…Recently, Castillo et al [33] evaluated the impact of photo retargeting on human fixations, by experimenting on the RetargetMe data set [19]. The authors observed that: 1) even strong artifacts in the retargeted photo cannot influence human gaze shifting if they are distributed outside the regions of interest; 2) removing contents in photo retargeting might change its semantics, which influences human perception of photo esthetics accordingly; and 3) employing eye-tracking data can more accurately reflect the regions of interest, which might be helpful for photo retargeting.…”

Section: Hmentioning

confidence: 99%

See 1 more Smart Citation

Perceptually Guided Photo Retargeting

Xia

Zhang

Hong

et al. 2017

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Abstract-We propose perceptually guided photo retargeting, which shrinks a photo by simulating a human's process of sequentially perceiving visually/semantically important regions in a photo. In particular, we first project the local features (graphlets in this paper) onto a semantic space, wherein visual cues such as global spatial layout and rough geometric context are exploited. Thereafter, a sparsity-constrained learning algorithm is derived to select semantically representative graphlets of a photo, and the selecting process can be interpreted by a path which simulates how a human actively perceives semantics in a photo. Furthermore, we learn the prior distribution of such active graphlet paths (AGPs) from training photos that are marked as esthetically pleasing by multiple users. The learned priors enforce the corresponding AGP of a retargeted photo to be maximally similar to those from the training photos. On top of the retargeting model, we further design an online learning scheme to incrementally update the model with new photos that are esthetically pleasing. The online update module makes the algorithm less dependent on the number and contents of the initial training data. Experimental results show that: 1) the proposed AGP is over 90% consistent with human gaze shifting path, as verified by the eye-tracking data, and 2) the retargeting algorithm outperforms its competitors significantly, as AGP is more indicative of photo esthetics than conventional saliency maps.

show abstract

Section: Hsupporting

confidence: 84%

Section: Hmentioning

confidence: 99%

Perceptually Guided Photo Retargeting

Xia

Zhang

Hong

et al. 2017

IEEE Trans. Cybern.

View full text Add to dashboard Cite

show abstract

“…The integration of their model could improve our visual importance evaluation algorithm and make it more consistent with human eyes. Castillo et al [44] examined the effect of the retargeting process on human fixations by gathering eye-tracking data for a representative benchmark of retargeting images. This scheme can also be employed to evaluate the rationality and quality of our OC-enhanced resizing method.…”

Section: Limitations and Discussionmentioning

confidence: 99%

Summarization-Based Image Resizing by Intelligent Object Carving

Dong

Zhou

Lee

et al. 2014

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Abstract-Image resizing can be more effectively achieved with a better understanding of image semantics. In this paper, similar patterns that exist in many real-world images are analyzed. By interactively detecting similar objects in an image, the image content can be summarized rather than simply distorted or cropped. This method enables the manipulation of image pixels or patches as well as semantic objects in the scene during image resizing process. Given the special nature of similar objects in a general image, the integration of a novel object carving (OC) operator with the multi-operator framework is proposed for summarizing similar objects. The object removal sequence in the summarization strategy directly affects resizing quality. The method by which to evaluate the visual importance of the object as well as to optimally select the candidates for object carving is demonstrated. To achieve practical resizing applications for general images, a template matching-based method is developed. This method can detect similar objects even when they are of various colors, transformed in terms of perspective, or partially occluded. To validate the proposed method, comparisons with state-of-the-art resizing techniques and a user study were conducted. Convincing visual results are shown to demonstrate the effectiveness of the proposed method.

show abstract

“…A different subjective study was proposed in [16], in which the user evaluation was carried out by simultaneous double stimulus for continuous evaluation [1] that scored only one retargeted image each time rather than pairwise comparison. Castillo et al [3] developed an image retargeting survey using eye tracking technology. All these subjective methods can provide good evaluation, but they are laborious and very time-consuming.…”

Section: Related Workmentioning

confidence: 99%

Learning to Rank Retargeted Images

Yang

Liu

Lai

2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Image retargeting techniques that adjust images into different sizes have attracted much attention recently. Objective quality assessment (OQA) of image retargeting results is often desired to automatically select the best results. Existing OQA methods output an absolute score for each retargeted image and use these scores to compare different results. Observing that it is challenging even for human subjects to give consistent scores for retargeting results of different source images, in this paper we propose a learningbased OQA method that predicts the ranking of a set of retargeted images with the same source image. We show that this more manageable task helps achieve more consistent prediction to human preference and is sufficient for most application scenarios. To compute the ranking, we propose a simple yet efficient machine learning framework that uses a General Regression Neural Network (GRNN) to model a combination of seven elaborate OQA metrics. We then propose a simple scheme to transform the relative scores output from GRNN into a global ranking. We train our GRNN model using human preference data collected in the elaborate RetargetMe benchmark and evaluate our method based on the subjective study in RetargetMe. Moreover, we introduce a further subjective benchmark to evaluate the generalizability of different OQA methods. Experimental results demonstrate that our method outperforms eight representative OQA methods in ranking prediction and has better generalizability to different datasets.

show abstract

Using eye-tracking to assess different image retargeting methods

Cited by 46 publications

References 28 publications

Perceptually Guided Photo Retargeting

Perceptually Guided Photo Retargeting

Summarization-Based Image Resizing by Intelligent Object Carving

Learning to Rank Retargeted Images

Contact Info

Product

Resources

About