Active Video Summarization: Customized Summaries via On-line Interaction with the User

Abstract

To facilitate the browsing of long videos, automatic video summarization provides an excerpt that represents its content. In the case of egocentric and consumer videos, due to their personal nature, adapting the summary to specific user's preferences is desirable. Current approaches to customizable video summarization obtain the user's preferences prior to the summarization process. As a result, the user needs to manually modify the summary to further meet the preferences. In this paper, we introduce Active Video Summarization (AVS), an interactive approach to gather the user's preferences while creating the summary. AVS asks questions about the summary to update it on-line until the user is satisfied. To minimize the interaction, the best segment to inquire next is inferred from the previous feedback. We evaluate AVS in the commonly used UTEgo dataset. We also introduce a new dataset for customized video summarization (CSumm) recorded with a Google Glass. The results show that AVS achieves an excellent compromise between usability and quality. In 41% of the videos, AVS is considered the best over all tested baselines, including summaries manually generated. Also, when looking for specific events in the video, AVS provides an average level of satisfaction higher than those of all other baselines after only six questions to the user.

FastPerson: Enhancing Video-Based Learning through Video Summarization that Preserves Linguistic and Visual Contexts

“…Learner-preference-based methods include many proposed video summarization methods that consider interaction history and user preference information [1,12]. There are methods for generating summaries based on user interaction data, such as Lecturescape [25], which retrieves frames frequently viewed by other learners, and EpicPlay, which identifies key moments in sports videos by leveraging social media viewer interaction information [48].…”

Personalized Video Summarization: A Comprehensive Survey of Methods and Datasets

“…• Dynamic summaries are also called dynamic video skims. Personalized video summarization is performed by selecting a subset of consecutive frames from all frames of the original video that includes the most relevant sub-shots that represent the original video according to the user's preferences as well as the summary criteria [12,14,[17][18][19][20]25,27,28,30,31,35,37,38,44,45,47,49,54,[59][60][61][62][63][64][65][66][67][68][69]. • A Hierarchical summary is a multilevel and scalable summary.…”

“…In the literature, there are several studies in which the creation of a personalized video summary is based on the user's query, which is taken into consideration [12,13,19,30,47,51,54,63,64,[67][68][69]72]. Sachan and Keshaveni [8] proposed a personalized video summarization approach using classificationbased entity identification.…”

Show Me What I Like: Detecting User-Specific Video Highlights Using Content-Based Multi-Head Attention

“…Other approaches for video summarization define and estimate specific parameters determining summaries, such as coherence [30], diversity, and representativeness [33,62]. Yet other approaches consider user preferences in the form of the text queries they use to search videos [29,38,44,52], their feedback on individual summaries [10], and their preferences between pairs of summaries [39], to learn user-specific summaries. Highlight detection methods learn to leverage video contents using techniques similar to those employed for video summarization, albeit with different training objectives.…”