Automatic Image Cropping for Visual Aesthetic Enhancement Using Deep Neural Networks and Cascaded Regression

Abstract

Abstract-Despite recent progress, computational visual aesthetic is still challenging. Image cropping, which refers to the removal of unwanted scene areas, is an important step to improve the aesthetic quality of an image. However, it is challenging to evaluate whether cropping leads to aesthetically pleasing results because the assessment is typically subjective. In this paper, we propose a novel cascaded cropping regression (CCR) method to perform image cropping by learning the knowledge from professional photographers. The proposed CCR method improves the convergence speed of the cascaded method, which directly uses random-ferns regressors. In addition, a two-step learning strategy is proposed and used in the CCR method to address the problem of lacking labelled cropping data. Specifically, a deep convolutional neural network (CNN) classifier is first trained on large-scale visual aesthetic datasets. The deep CNN model is then designed to extract features from several image cropping datasets, upon which the cropping bounding boxes are predicted by the proposed CCR method. Experimental results on public image cropping datasets demonstrate that the proposed method significantly outperforms several state-of-the-art image cropping methods.

Reliable and Efficient Image Cropping: A Grid Anchor Based Approach

“…Though a number of image cropping methods have been developed [34,11,5,6,10,15,22,36], many of them do not release the source code or executable program. We thus compare our method, namely Grid Anchor based Image Cropping (GAIC), with the following baseline and recently developed state-of-the-art methods whose source codes are available.…”

“…Most recent methods are datadriven, which train an end-to-end CNN model for image cropping. However, limited by the insufficient number of annotated training samples, many methods in this category [5,34,35,11,10,15,22] adopt a general aesthetic classifier trained from image aesthetic databases such as AVA [28] and CUHKPQ [25] to help cropping. However, a general aesthetic classifier trained on full images may not be able to reliably evaluate the crops within one image [6,36].…”

“…Feature extraction: As in many previous works [34,11,5,6,10,15,22,36], we truncate one pre-trained CNN model (e.g., VGG16 [31] or ResNet50 [17]) as the feature extraction module. The spatial arrangement of context and objects in an image plays a key role in image composition.…”

“…Two objective metrics, namely intersection-overunion (IoU) and boundary displacement error (BDE) [14], were defined to evaluate the performance of image cropping models on these databases. These public benchmarks enable many researchers to develop and test their cropping models, significantly facilitating the research on automatic image cropping [39,11,34,5,6,10,15,22,36]. Though many efforts have been made, there exists sev- Table 1.…”

A fast face detection method via convolutional neural network

“…Deep learning models have demonstrated impressive performance for different computer vision applications [6,7,8,9,10,11,12]. The deep convolutional neural network (CNN) [13] can map raw data from a manifold to the Euclidean space, in which features may be linearly separable.…”