Automatic Image Cropping for Visual Aesthetic Enhancement Using Deep Neural Networks and Cascaded Regression

Guo, Guanjun; Wang, Hanzi; Shen, Chunhua; Yan, Yan; Liao, Hong-Yuan Mark

doi:10.48550/arxiv.1712.09048

Cited by 3 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most recent methods are data-driven, which train an end-to-end CNN model for image cropping. Limited by the insufficient number of annotated training samples, many methods in this category [10], [11], [13], [20], [22], [23], [25] adopt a general aesthetic classifier trained from image aesthetic databases such as AVA [38] and CUHKPQ [39] to help cropping. However, a general aesthetic classifier trained on full images may not be able to reliably evaluate the crops within one image [21], [24].…”

Section: Image Cropping Methodsmentioning

confidence: 99%

“…Though a number of image cropping methods have been developed [10], [11], [13], [20], [21], [22], [23], [24], many of them do not release the source code or executable program. We thus compare our method, namely Grid Anchor based Image Cropping (GAIC), with the following baseline and recently developed stateof-the-art methods whose source codes are available.…”

Section: Comparison Methodsmentioning

confidence: 99%

“…Efficient base model: A practical cropping model needs to be lightweight and efficient enough to be deployed on resource limited devices. Instead of employing those classical pre-trained CNN models such as AlexNet [47], VGG16 [29] or ResNet50 [30] as in previous work [10], [11], [13], [20], [21], [22], [23], [24], [28], we choose the more efficient architectures including the MobileNetV2 [26] and ShuffleNetV2 [27]. Fortunately, we found that using such efficient models will not sacrifice the cropping accuracy compared with their complicated counterparts, mostly owing to the special properties of image cropping and more advanced architecture designs of the MobileNetV2 and ShuffleNetV2.…”

Section: Multi-scale Feature Extraction Modulementioning

confidence: 99%

See 2 more Smart Citations

Grid Anchor based Image Cropping: A New Benchmark and An Efficient Model

Zhong¹,

Li²,

Cao³

et al. 2019

Preprint

View full text Add to dashboard Cite

Image cropping aims to improve the composition as well as aesthetic quality of an image by removing extraneous content from it. Most of the existing image cropping databases provide only one or several human-annotated bounding boxes as the groundtruths, which can hardly reflect the non-uniqueness and flexibility of image cropping in practice. The employed evaluation metrics such as intersection-over-union cannot reliably reflect the real performance of a cropping model, either. This work revisits the problem of image cropping, and presents a grid anchor based formulation by considering the special properties and requirements (e.g., local redundancy, content preservation, aspect ratio) of image cropping. Our formulation reduces the searching space of candidate crops from millions to no more than ninety. Consequently, a grid anchor based cropping benchmark is constructed, where all crops of each image are annotated and more reliable evaluation metrics are defined. To meet the practical demands of robust performance and high efficiency, we also design an effective and lightweight cropping model. By simultaneously considering the region of interest and region of discard, and leveraging multi-scale information, our model can robustly output visually pleasing crops for images of different scenes. With less than 2.5M parameters, our model runs at a speed of 200 FPS on one single GTX 1080Ti GPU and 12 FPS on one i7-6800K CPU.

show abstract

Section: Image Cropping Methodsmentioning

confidence: 99%

Section: Comparison Methodsmentioning

confidence: 99%

Section: Multi-scale Feature Extraction Modulementioning

confidence: 99%

See 1 more Smart Citation

Grid Anchor based Image Cropping: A New Benchmark and An Efficient Model

Zhong¹,

Li²,

Cao³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Most recent methods are datadriven, which train an end-to-end CNN model for image cropping. However, limited by the insufficient number of annotated training samples, many methods in this category [5,34,35,11,10,15,22] adopt a general aesthetic classifier trained from image aesthetic databases such as AVA [28] and CUHKPQ [25] to help cropping. However, a general aesthetic classifier trained on full images may not be able to reliably evaluate the crops within one image [6,36].…”

Section: Related Workmentioning

confidence: 99%

“…Two objective metrics, namely intersection-overunion (IoU) and boundary displacement error (BDE) [14], were defined to evaluate the performance of image cropping models on these databases. These public benchmarks enable many researchers to develop and test their cropping models, significantly facilitating the research on automatic image cropping [39,11,34,5,6,10,15,22,36].…”

Section: Introductionmentioning

confidence: 99%

Reliable and Efficient Image Cropping: A Grid Anchor Based Approach

Zeng

Cao

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Image cropping aims to improve the composition as well as aesthetic quality of an image by removing extraneous content from it. Existing image cropping databases provide only one or several human-annotated bounding boxes as the groundtruth, which cannot reflect the non-uniqueness and flexibility of image cropping in practice. The employed evaluation metrics such as intersection-over-union cannot reliably reflect the real performance of cropping models, either. This work revisits the problem of image cropping, and presents a grid anchor based formulation by considering the special properties and requirements (e.g., local redundancy, content preservation, aspect ratio) of image cropping. Our formulation reduces the searching space of candidate crops from millions to less than one hundred. Consequently, a grid anchor based cropping benchmark is constructed, where all crops of each image are annotated and more reliable evaluation metrics are defined. We also design an effective and lightweight network module, which simultaneously considers the region of interest and region of discard for more accurate image cropping. Our model can stably output visually pleasing crops for images of different scenes and run at a speed of 125 FPS. Code and dataset are available at: https://github.com/HuiZeng/ Grid-Anchor-based-Image-Cropping.

show abstract