"My Day in Review"

Abstract

Lifelogging devices, which seamlessly gather various data about a user as they go about their daily life, have resulted in users amassing large collections of noisy photographs (e.g. visual duplicates, image blur), which are difficult to navigate, especially if they want to review their day in photographs. Social media websites, such as Facebook, have faced a similar information overload problem for which a number of summarization methods have been proposed (e.g. news story clustering, comment ranking etc.). In particular, Facebook's 'Year in Review' received much user interest where the objective for the model was to identify key moments in a user's year, offering an automatic visual summary based on their uploaded content. In this paper, we follow this notion by automatically creating a review of a user's day using lifelogging images. Specifically, we address the quality issues faced by the photographs taken on lifelogging devices and attempt to create visual summaries by promoting visual and temporalspatial diversity in the top ranks. Conducting two crowdsourced evaluations based on 9k images, we show the merits of combining time, location and visual appearance for summarization purposes.

Overview of Lifelogging: Current Challenges and Advances

“…taken whilst on the move, there is a problem of blurriness. In fact, the blurred images will not provide enough information; yet reduce the efficiency of search performance due to the wasted computation time [59], [60]. The third issue is about images redundancy.…”

“…The third issue is about images redundancy. Since the lifelogger may be in stationary situations during the day, duplicates photos tend to exist within the lifelogs [59]. The retrieval of such images is time consuming without any benefit.…”

“…Soumyadeb et al estimated Blur score using a Haar wavelet transform. Images below the threshold are pruned [59]. The UPB team [60] deals with the uninformativeness and blurriness.…”