Learning to Select and Order Vacation Photographs

Abstract

We propose the problem of automated photo album creation from an unordered image collection. The problem is difficult as it involves a number of complex perceptual tasks that facilitate selection and ordering of photos to create a compelling visual narrative. To help solve this problem, we collect (and will make available) a new benchmark dataset based on Flickr images. Flickr Album Dataset and provides a variety of annotations useful for the task, including manually created albums of various lengths. We analyze the problem and provide experimental evidence, through user studies, that both selection and ordering of photos within an album is important for human observers. To capture and learn rules of album composition, we propose a discriminative structured model capable of encoding simple preferences for contextual layout of the scene (e.g., spatial layout of faces, global scene context, and presence/absence of attributes) and ordering between photos (e.g., exclusion principles or correlations). The parameters of the model are learned using a structured SVM framework. Once learned, the model allows automatic composition of photo albums from unordered and untagged collections of images. We quantitatively evaluate the results obtained using our model against manually created albums and baselines on a dataset of 63 personal photo collections from 5 different topics.

Learning Visual Storylines with Skipping Recurrent Neural Networks

“…Sadeghi et al . [28] also consider if a photo is memorable or iconic. Moreover, most of these approaches are supervised, namely the associated summaries for videos/albums are first collected by crowd-sourcing, then a model is learned to generate good summaries.…”

“…A typical example is a series of photos, taken by a family on their visit to Paris, visiting all the iconic landmarks, such as the Eiffel Tower. Classically, summarization is approached by collecting a dataset of videos/albums and their associated summaries generated by people [26,[28][29][30][31], in order to learn how to make a summary in a supervised way. This process is, however, considerably laborious.…”

Deep Multimodal Feature Encoding for Video Ordering

“…Similar in spirit to our temporal ordering of video clips, arranging photos by time has also received some attention. In particular, automating the process of creating temporally ordered photo albums from an unordered image collection [46]; sorting photo collections spanning many years [30,48]; or ordering a set of photos taken from uncalibrated cameras [4,7,33] are all related works. In contrast, we work with an unordered collection of video clips.…”

Joint optimisation convex‐negative matrix factorisation for multi‐modal image collection summarisation based on images and tags

“…The output is the multi-modal summary, S, which contains a visual summary and a textual topic. First step (1)(2)(3)(4)(5)(6)(7)(8): Equation (1) is optimised to produce (9)(10)(11)(12)(13)(14)(15): W t and W v are used to find the multi-modal summary, S, which includes a visual summary, ψ, and textual topic, ϕ. For the textual topic, F t captures the notion of centroids that represent a high-level semantic feature.…”

“…The small subset displays the various aspects of the original collection including the target attribute and scene type. Various multimedia applications can benefit from image collection summarisation including automated album creation [1], search results improvement [2] etc.…”