Attention-based Multi-Patch Aggregation for Image Aesthetic Assessment

Sheng, Kekai; Dong, Weiming; Ma, Chongyang; Mei, Xing; Huang, Feiyue; Hu, Bao-Gang

doi:10.1145/3240508.3240554

Cited by 101 publications

(72 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…With the renaissance of deep learning, CNN has been widely used in recent works [10,20,22,30,32,35,48] and achieves promising results. From the perspective of model inputs, most methods use image patches as a part of their model inputs.…”

Section: Image Aesthetic Assessment (Iaa)mentioning

confidence: 99%

“…Early deep learning based IAA methods [19,20] comprised each input by combining a resized image with several small patches randomly cropped from the resized image. Due to the effectiveness, many later methods [10,22,30,32,48] adopted a similar strategy and utilized one or multiple patches selected from holistic images as part of the model input. However, for the human visual system, patches selected from different regions contribute differently to overall aesthetics, which forms a natural requirement for selectively concentrating on more contributive regions when inferring aesthetic values.…”

mentioning

confidence: 99%

“…Recent methods address this issue by adopting attention mechanisms. Sheng et al [30] adopted attention mechanism to dynamically adjust the weight on each randomly selected patch during training. Zhang et al [48] proposed a two-stream model that takes a global view and a local view as input, where the local view is selected according to pixel-level spatial attention learned from the global view.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Object-level Attention for Aesthetic Rating Distribution Prediction

Hou

Yang

Wang

2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

We study the problem of image aesthetic assessment (IAA) and aim to automatically predict the image aesthetic quality in the form of discrete distribution, which is particularly important in IAA due to its nature of having possibly higher diversification of agreement for aesthetics. Previous works show the effectiveness of utilizing objectagnostic attention mechanisms to selectively concentrate on more contributive regions for IAA, e.g., attention is learned to weight pixels of input images when inferring aesthetic values. However, as suggested by some neuropsychology studies, the basic units of human attention are visual objects, i.e., the trace of human attention follows a series of objects. This inspires us to predict contributions of different regions at object level for better aesthetics evaluation. With our framework, region-of-interests (RoIs) are proposed by an object detector, and each RoI is associated with a regional feature vector. Then the contribution of each regional feature to the aesthetics prediction is adaptively determined. To the best of our knowledge, this is the first work modeling object-level attention for IAA and experimental results confirm the superiority of our framework over previous relevant methods. CCS CONCEPTS • Applied computing → Media arts; • Computing methodologies → Computer vision; Neural networks.

show abstract

Section: Image Aesthetic Assessment (Iaa)mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Object-level Attention for Aesthetic Rating Distribution Prediction

Hou

Yang

Wang

2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…In the existing aesthetic computing research, there are three main types of studies, the aesthetic ranking analysis [13], classification of aesthetic level (low/high or positive/negative) [15,18,24,[26][27][28][29] and the aesthetic score prediction [17,31,43]. In the majority of the related works, researchers conducted classification method on image aesthetic computing study.…”

Section: Methodologiesmentioning

confidence: 99%

“…This can be an extremely important evidence for the study of features of visual perception. Thus, the aesthetic principles and patterns of multimedia works should be explored using computer models [12][13][14][15]. Here we review the related works of multimedia aesthetic computing, and the existing multimedia aesthetic databases are concluded.…”

Section: Related Workmentioning

confidence: 99%

And the nominees are: Using design-awards datasets to build computational aesthetic evaluation model

Xing

Zhang

et al. 2020

PLoS ONE

View full text Add to dashboard Cite

Aesthetic perception is a human instinct that is responsive to multimedia stimuli. Giving computers the ability to assess human sensory and perceptual experience of aesthetics is a well-recognized need for the intelligent design industry and multimedia intelligence study. In this work, we constructed a novel database for the aesthetic evaluation of design, using 2,918 images collected from the archives of two major design awards, and we also present a method of aesthetic evaluation that uses machine learning algorithms. Reviewers' ratings of the design works are set as the ground-truth annotations for the dataset. Furthermore, multiple image features are extracted and fused. The experimental results demonstrate the validity of the proposed approach. Primary screening using aesthetic computing can be an intelligent assistant for various design evaluations and can reduce misjudgment in art and design review due to visual aesthetic fatigue after a long period of viewing. The study of computational aesthetic evaluation can provide positive effect on the efficiency of design review, and it is of great significance to aesthetic recognition exploration and applications development.

show abstract

References

2023

Aesthetics in Digital Photography

View full text Add to dashboard Cite

Attention-based Multi-Patch Aggregation for Image Aesthetic Assessment

Cited by 101 publications

References 21 publications

Object-level Attention for Aesthetic Rating Distribution Prediction

Object-level Attention for Aesthetic Rating Distribution Prediction

And the nominees are: Using design-awards datasets to build computational aesthetic evaluation model

References

Contact Info

Product

Resources

About