Hierarchical Layout-Aware Graph Convolutional Network for Unified Aesthetics Assessment

She, Dongyu; Lai, Yu‐Kun; Yi, Gaoxiong; Xu, Kun

doi:10.1109/cvpr46437.2021.00837

Cited by 64 publications

(51 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although abundant deep learning methods [75,134,21,34,101,92,126] have achieved significant progress on image aesthetic evaluation, only a few of them [101,92,126] can be viewed as composition-aware methods which explicitly model image composition. These composition-aware methods focus on learning the relations among the visual elements (regions or objects) in the image.…”

Section: Deep Learning Methodsmentioning

confidence: 99%

“…The classification accuracy is computed between the predicted and ground-truth binary labels of aesthetic quality, and MSE measures the error between the predicted and ground-truth aesthetic mean scores (mean opinion scores). Then, to measure the ranking correlation and linear association between the estimated and ground-truth aesthetic scores, Spearman's Rank Correlation Coefficient (SRCC) and Linear Correlation Coefficient (LCC) are used in recent aesthetic evaluation approaches [75,134,21,126]. Additionally, for these methods [134,21,59,126] that are capable of producing the prediction of aesthetic score distribution, Earth Mover's Distance (EMD) [59] is also frequently utilized to measure the closeness between the predicted and ground-truth composition score distributions.…”

Section: A Datasets and Evaluation Metricsmentioning

confidence: 99%

“…In this case, we need to further evaluate the aesthetic value of a composite image. There have been many works on aesthetic evaluation [134,21,92,126], which can judge an image from multiple aesthetic aspects (e.g., interesting lighting, harmony, vivid color, depth of field, rule of thirds, symmetry). In this paper, we focus on compositionaware aesthetic evaluation (e.g., rule of thirds, symmetry), because we can adjust the location and size of foreground to achieve higher composition quality when compositing an image.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Making Images Real Again: A Comprehensive Survey on Deep Image Composition

Niu¹,

Cong²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

As a common image editing operation, image composition aims to cut the foreground from one image and paste it on another image, resulting in a composite image. However, there are many issues that could make the composite images unrealistic. These issues can be summarized as the inconsistency between foreground and background, which include appearance inconsistency (e.g., incompatible color and illumination) and geometry inconsistency (e.g., unreasonable size and location). Previous works on image composition target at one or more issues. Since each individual issue is a complicated problem, there are some research directions (e.g., image harmonization, object placement) which focus on only one issue. By putting all the efforts together, we can acquire realistic composite images. Sometimes, we expect the composite images to be not only realistic but also aesthetic, in which case aesthetic evaluation needs to be considered. In this survey, we summarize the datasets and methods for the above research directions. We also discuss the limitations and potential directions to facilitate the future research for image composition. Finally, as a double-edged sword, image composition may also have negative effect on our lives (e.g., fake news) and thus it is imperative to develop algorithms to fight against composite images. Datasets and codes for image composition are summarized at https://github.com/bcmi/Awesome-Image-Composition.

show abstract

Section: Deep Learning Methodsmentioning

confidence: 99%

Section: A Datasets and Evaluation Metricsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Making Images Real Again: A Comprehensive Survey on Deep Image Composition

Niu¹,

Cong²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, they are not trained on target datasets, which might not always achieve acceptable performance. Alternatively, an input image can be directly split into grids and take each grid cell as a region [80,120].…”

Section: Cnn-based Cpamentioning

confidence: 99%

“…To obtain the final predictions from the regions, [80,120] proposed to use graph convolutional operations to reason on the graph of ROIs. Each node in the graph represents an ROI, while each edge represents the similarity between two ROIs.…”

Section: Cnn-based Cpamentioning

confidence: 99%

AI-Empowered Persuasive Video Generation: A Survey

Liu¹,

Han²

2021

Preprint

View full text Add to dashboard Cite

Promotional videos are rapidly becoming a popular medium for persuading people to change their behaviours in many settings (e.g., online shopping, social enterprise initiatives). Today, such videos are often produced by professionals, which is a time-, labour-and cost-intensive undertaking. In order to produce such contents to support a large applications (e.g., e-commerce), the field of artificial intelligence (AI)-empowered persuasive video generation (AIPVG) has gained traction in recent years. This field is interdisciplinary in nature, which makes it challenging for new researchers to grasp. Currently, there is no comprehensive survey of AIPVG available. In this paper, we bridge this gap by reviewing key AI techniques that can be utilized to automatically generate persuasive videos. We offer a first-of-its-kind taxonomy which divides AIPVG into three major steps: 1) visual material understanding, which extracts information from the visual materials (VMs) relevant to the target of promotion; 2) visual storyline generation, which shortlists and arranges high-quality VMs into a sequence in order to compose a storyline with persuasive power; and 3) post-production, which involves background music generation and still image animation to enhance viewing experience. We also introduce the evaluation metrics and datasets commonly adopted in the field of AIPVG. We analyze the advantages and disadvantages of the existing works belonging to the above-mentioned steps, and discuss interesting potential future research directions.

show abstract