Visual Summarization of Lecture Video Segments for Enhanced Navigation

Abstract

Lecture videos are an increasingly important learning resource for higher education. However, the challenge of quickly finding the content of interest in a lecture video is an important limitation of this format. This paper introduces visual summarization of lecture video segments to enhance navigation. A lecture video is divided into segments based on the frame-to-frame similarity of content. The user navigates the lecture video content by viewing a single frame visual and textual summary of each segment. The paper presents a novel methodology to generate the visual summary of a lecture video segment by computing similarities between images extracted from the segment and employing a graph-based algorithm to identify the subset of most representative images. The results from this research are integrated into a real-world lecture video management portal called Videopoints. To collect ground truth for evaluation, a survey was conducted where multiple users manually provided visual summaries for 40 lecture video segments selected from a diverse set of courses. The users also stated whether any images were not selected for the summary because they were similar to other selected images. The graph based algorithm for identifying summary images achieves 78% precision and 72% F1-measure with frequently selected images as the ground truth, and 94% precision and 72% F1-measure with the union of all user selected images as the ground truth. For 98% of algorithm selected visual summary images, at least one user also selected that image for their summary or considered it similar to another image they selected. Over 65% of automatically generated summaries were rated as good or very good by the users on a 4-point scale from poor to very good. Overall, the results establish that the methodology introduced in this paper produces good quality visual summaries that are practically useful for lecture video navigation.

FCN-LectureNet: Extractive Summarization of Whiteboard and Chalkboard Lecture Videos

“…For slide-based videos, the extraction of the content is usually done by simply relying on out-of-the-box OCR systems [33], [34]. However, slides are different from both scanned documents and scene text images and some methods rely on training end-to-end detection and recognition systems.…”

An Efficient Projection Screen Detection from Presentation Videos using CustomYOLOv7 Object Detection Method

“…Lecture videos are analyzed for the development of various applications that utilize the content of lecture videos, such as indexing [1][2][3][4][5], summarization [6][7][8][9][10][11], content extraction [12][13][14][15][16][17], search [18][19][20][21], and navigation [22][23][24][25]. Lecture videos captured in classrooms and conference rooms has digital slides projected on to the screen on stage, a common setup in modern classrooms and conference rooms.…”

Lecture Video Automatic Summarization System Based on DBNet and Kalman Filtering

“…Conflicts between regions are time-segmented to extract key frames, and tests on the AccessMath dataset show that the summary method has a good compression ratio. Rahman et al [7] proposed a new visual summarization method for lecture videos by dividing the video into multiple segments based on the inter-frame similarity of the content and defining the most representative images by estimating the importance of each image in the segment, calculating the distance matrix between images, and using a graph-based algorithm; the proposed algorithm is significantly better than random selection and cluster-based selection, and only slightly lower than manual selection.…”