Reliable and Efficient Image Cropping: A Grid Anchor Based Approach

Zeng, Hui; Li, Lida; Cao, Zisheng; Zhang, Lei

doi:10.1109/cvpr.2019.00610

Cited by 62 publications

(107 citation statements)

References 36 publications

Supporting

Mentioning

104

Contrasting

Unclassified

Order By: Relevance

“…Following this idea, Wei et al (2018) increased the number of crops in each image and proposed the first densely annotated image cropping dataset, in which each image has 24 annotated crops with aesthetic scores. Zeng et al (2019) took a further step and presented another densely annotated dataset with about 85 crops for each image. In our method, we use such datasets to train our image cropping model.…”

Section: Related Workmentioning

confidence: 99%

“…Based on the aesthetic score map, we aim to find the best crop by generating a lot of candidate crops and comparing their crop-level scores. Previous methods (Chen et al 2017b;Wei et al 2018;Zeng et al 2019) adopt different strategies in this step. Here, we design a simple yet effective searching strategy to reduce the searching space.…”

Section: Two-stage Search For Image Croppingmentioning

confidence: 99%

See 1 more Smart Citation

Image Cropping with Composition and Saliency Aware Aesthetic Score Map

Niu

Zhao

et al. 2020

AAAI

View full text Add to dashboard Cite

Aesthetic image cropping is a practical but challenging task which aims at finding the best crops with the highest aesthetic quality in an image. Recently, many deep learning methods have been proposed to address this problem, but they did not reveal the intrinsic mechanism of aesthetic evaluation. In this paper, we propose an interpretable image cropping model to unveil the mystery. For each image, we use a fully convolutional network to produce an aesthetic score map, which is shared among all candidate crops during crop-level aesthetic evaluation. Then, we require the aesthetic score map to be both composition-aware and saliency-aware. In particular, the same region is assigned with different aesthetic scores based on its relative positions in different crops. Moreover, a visually salient region is supposed to have more sensitive aesthetic scores so that our network can learn to place salient objects at more proper positions. Such an aesthetic score map can be used to localize aesthetically important regions in an image, which sheds light on the composition rules learned by our model. We show the competitive performance of our model in the image cropping task on several benchmark datasets, and also demonstrate its generality in real-world applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Two-stage Search For Image Croppingmentioning

confidence: 99%

Image Cropping with Composition and Saliency Aware Aesthetic Score Map

Niu

Zhao

et al. 2020

AAAI

View full text Add to dashboard Cite

show abstract

“…We show the retargeting results of our method and some previous methods in Figure 4 and Figure 5, where the retargeted size is 50% of the original size. The previous methods; scaling, GAIC [33], seam carving [21], MULTIOP [22], WSSDCNN [3], and Cycle-IR [26] were compared. The retargeting operators of scaling and seam carving were the same as the operators used in our method.…”

Section: Qualitative Evaluationmentioning

confidence: 99%

“…This trend became more pronounced as the number of operators increased. Figure 7: Qualitative comparison when the aesthetic score was used as a reward to generate 75% retargeted images : (a) original image, (b) SCL, (c) GAIC [33], (d) SC [21], (e) MULTIOP [22], (f) Our method, (g) Our method without both the self-play-based reward and the frequency-aware weighted loss, (h) Our method without the frequency-aware weighted loss. The number below each image represents the aesthetic score; a higher score means a more aesthetic image.…”

Section: Time Efficiencymentioning

confidence: 99%

Self-Play Reinforcement Learning for Fast Image Retargeting

Kajiura

Kosugi

Wang

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

In this study, we address image retargeting, which is a task that adjusts input images to arbitrary sizes. In one of the best-performing methods called MULTIOP, multiple retargeting operators were combined and retargeted images at each stage were generated to find the optimal sequence of operators that minimized the distance between original and retargeted images. The limitation of this method is in its tremendous processing time, which severely prohibits its practical use. Therefore, the purpose of this study is to find the optimal combination of operators within a reasonable processing time; we propose a method of predicting the optimal operator for each step using a reinforcement learning agent. The technical contributions of this study are as follows. Firstly, we propose a reward based on self-play, which will be insensitive to the large variance in the content-dependent distance measured in MULTIOP. Secondly, we propose to dynamically change the loss weight for each action to prevent the algorithm from falling into a local optimum and from choosing only the most frequently used operator in its training. Our experiments showed that we achieved multi-operator image retargeting with less processing time by three orders of magnitude and the same quality as the original multi-operator-based method, which was the best-performing algorithm in retargeting tasks. CCS CONCEPTS • Computing methodologies → Reinforcement learning; Image processing; Neural networks.

show abstract

“…The preprocessing stage consists of two parts: rescaling and obtaining optical flow maps of each eight consecutive video frames. The universal video format of today is 16:9 [26], which is an aspect ratio with a width of 16 units and a height of 9 units. In our study, each frame was extracted from the dataset videos and rescaled to 320 x 180.…”

Section: A Preprocessingmentioning

confidence: 99%

Anomaly Detection in Videos Using Optical Flow and Convolutional Autoencoder

Duman

Erdem

2019

IEEE Access

View full text Add to dashboard Cite

Today, public areas, such as airports, hospitals, city centers are monitored by surveillance systems. The widespread use of surveillance systems reduces security concerns while creating an amount of video data that cannot be examined by people in real-time. Therefore, the concept of automatic understanding of video activities has raised the standards of security camera systems. In this paper, we propose a framework (OF-ConvAE-LSTM) to detect anomalies using Convolutional Autoencoder and Convolutional Long Short-Term Memory in an unsupervised manner. Besides the deep learning model, the feature extraction stage based on dense optical flow is applied in the framework to obtain the velocity and direction information of foreground objects. The experiments were carried out on three popular public datasets consisting of Avenue, UCSD Ped1, and UCSD Peds2. The experimental results have shown that the proposed framework models the complex distribution of the pattern of regular motion changes with high accuracy. Besides, this method was observed to outperform state-of-the-art approaches based on unsupervised and semi-supervised deep learning models.INDEX TERMS Abnormal event detection, convolutional autoencoder, long short-term memory, optical flow.

show abstract

Reliable and Efficient Image Cropping: A Grid Anchor Based Approach

Cited by 62 publications

References 36 publications

Image Cropping with Composition and Saliency Aware Aesthetic Score Map

Image Cropping with Composition and Saliency Aware Aesthetic Score Map

Self-Play Reinforcement Learning for Fast Image Retargeting

Anomaly Detection in Videos Using Optical Flow and Convolutional Autoencoder

Contact Info

Product

Resources

About