Towards a comprehensive computational model foraesthetic assessment of videos

Abstract

In this paper we propose a novel aesthetic model emphasizing psychovisual statistics extracted from multiple levels in contrast to earlier approaches that rely only on descriptors suited for image recognition or based on photographic principles. At the lowest level, we determine dark-channel, sharpness and eye-sensitivity statistics over rectangular cells within a frame. At the next level, we extract Sentibank features (1, 200 pre-trained visual classifiers) on a given frame, that invoke specific sentiments such as "colorful clouds", "smiling face" etc. and collect the classifier responses as framelevel statistics. At the topmost level, we extract trajectories from video shots. Using viewer's fixation priors, the trajectories are labeled as foreground, and background/camera on which statistics are computed. Additionally, spatio-temporal local binary patterns are computed that capture texture variations in a given shot. Classifiers are trained on individual feature representations independently. On thorough evaluation of 9 different types of features, we select the best features from each level -dark channel, affect and camera motion statistics. Next, corresponding classifier scores are integrated in a sophisticated low-rank fusion framework to improve the final prediction scores. Our approach demonstrates strong correlation with human prediction on 1, 000 broadcast quality videos released by NHK as an aesthetic evaluation dataset.

Predicting Viewer Perceived Emotions in Animated GIFs

“…We perform face detection using OpenCV's Haar-like cascade and apply facial expression recognition on the largest face for a 6-D vector of SVM score outputs as a feature. Image-based aesthetics: Earlier works have shown that emotion has some intrinsic correlation with visual aesthetics [3,10,12]. We compute a subset of well-known image-based aesthetic features as described in [3].…”

Large-scale visual sentiment ontology and detectors using adjective noun pairs

“…Thirdly, we test aesthetics related features on all of the ANPs. These features from [41] include dark channel and luminosity feature, sharpness, symmetry, low depth of field, white balance, etc. The above three groups of features increase the mean AP@20 score of selected ANP sets by 10.5%, 13.5% and 9.0% (relative gains) respectively on the reduced testset and the mean AP@100 by 39.1%, 15.8% and 30.7% on the full testset.…”

Modelling the influence of personality and culture on affect and enjoyment in multimedia

“…Some focus on distilling the influence of specific cinematographic theories [11], types of segment and shot [12], the use of colour [13] and connotative space [14]. Others focus on modelling these different audio-visual features to predict emotions [15].…”