Semantic mosaic for indexing and compressing instructional videos

Liu, T.; Kender, John R.

doi:10.1109/icip.2003.1247114

Cited by 7 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, a body of research has been developed to analyze lecture videos. Several contributions focus on finding unique transition frames or minimizing the number of frames of videos across different types of presentations like powerpoint lectures and blackboard handwriting [10,25,29,37,38]. Recent research has also focused on methods for efficient access to educational videos with automatic identification of keywords from videos or their segments [31,36].…”

Section: Related Workmentioning

confidence: 99%

Visual Summarization of Lecture Video Segments for Enhanced Navigation

Rahman

Shah

Subhlok

2020

2020 IEEE International Symposium on Multimedia (ISM)

View full text Add to dashboard Cite

Lecture videos are an increasingly important learning resource for higher education. However, the challenge of quickly finding the content of interest in a lecture video is an important limitation of this format. This paper introduces visual summarization of lecture video segments to enhance navigation. A lecture video is divided into segments based on the frame-to-frame similarity of content. The user navigates the lecture video content by viewing a single frame visual and textual summary of each segment. The paper presents a novel methodology to generate the visual summary of a lecture video segment by computing similarities between images extracted from the segment and employing a graph-based algorithm to identify the subset of most representative images. The results from this research are integrated into a real-world lecture video management portal called Videopoints. To collect ground truth for evaluation, a survey was conducted where multiple users manually provided visual summaries for 40 lecture video segments selected from a diverse set of courses. The users also stated whether any images were not selected for the summary because they were similar to other selected images. The graph based algorithm for identifying summary images achieves 78% precision and 72% F1-measure with frequently selected images as the ground truth, and 94% precision and 72% F1-measure with the union of all user selected images as the ground truth. For 98% of algorithm selected visual summary images, at least one user also selected that image for their summary or considered it similar to another image they selected. Over 65% of automatically generated summaries were rated as good or very good by the users on a 4-point scale from poor to very good. Overall, the results establish that the methodology introduced in this paper produces good quality visual summaries that are practically useful for lecture video navigation.

show abstract

Section: Related Workmentioning

confidence: 99%

Visual Summarization of Lecture Video Segments for Enhanced Navigation

Rahman

Shah

Subhlok

2020

2020 IEEE International Symposium on Multimedia (ISM)

View full text Add to dashboard Cite

show abstract

“…The video summaries can be key frames [3,7,34], video segments condensed using skimming techniques [24,25], and the mosaicked images [1,10,16]. Because video key frames are compact visual summaries that allow users to quickly understand and access video content, in this paper, we focus on the key-frame-based methods and systems.…”

Section: Related Workmentioning

confidence: 99%

A two-level queueing system for interactive browsing and searching of video content

Liu

Katpelly

2006

Multimedia Systems

View full text Add to dashboard Cite

This paper presents a two-level queueing system for dynamic summarization and interactive searching of video content. Video frames enter the queueing system; some insignificant and redundant frames are removed; the remaining frames are pulled out of the system as top-level key frames. Using an energyminimization method, the first queue removes the video frames that constitute the gradual transitions of video shots. The second queue measures the content similarity of video frames and reduces redundant frames. In the queueing system, all key frames are linked in a directed-graph index structure, allowing video content to be accessed at any level-of-detail. Furthermore, this graph-based index structure enables interactive video content exploration, and the system is able to retrieve the video key frames that complement the video content already viewed by users. Experimental results on four full-length videos show that our queueing system performs much better than two existing methods on video key frame selection at different compression ratios. The evaluation on video content search shows that our interactive system is more effective than other systems on eight video searching tasks. Compared with the regular media player, our system reduces the average content searching time by half.

show abstract

“…Aner and Kender [2004] introduced a mosaic-based summarization approach to represent video scenes and applied it to situation comedy videos. An approach [Liu and Kender 2003] of semantic summarization was developed for instructional videos by stitching the textual content in these videos without resorting to pixel-level image mosaicking. In addition, motion features can be extracted for visualizing, summarizing, and searching video content [Chang et al 1997;Teodosio and Bender 2005].…”

Section: Introductionmentioning

confidence: 99%

Computational approaches to temporal sampling of video sequences

Liu

Kender

2007

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

Video key frame extraction is one of the most important research problems for video summarization, indexing, and retrieval. For a variety of applications such as ubiquitous media access and video streaming, the temporal boundaries between video key frames are required for synchronizing visual content with audio. In this article, we define temporal video sampling as a unified process of extracting video key frames and computing their temporal boundaries, and formulate it as an optimization problem. We first provide an optimal approach that minimizes temporal video sampling error using a dynamic programming process. The optimal approach retrieves a key frame hierarchy and all temporal boundaries in O(n 4 ) time and O(n 2 ) space. To further reduce computational complexity, we also provide a suboptimal greedy algorithm that exploits the data structure of a binary heap and uses a novel "look-ahead" computational technique, enabling all levels of key frames to be extracted with an average-case computational time of O(n log n) and memory usage of O(n). Both the optimal and the greedy methods are free of parameters, thus avoiding the threshold-selection problem that exists in other approaches. We empirically compare the proposed optimal and greedy methods with several existing methods in terms of video sampling error, computational cost, and subjective quality. An evaluation of eight videos of different genres shows that the greedy approach achieves performance very close to that of the optimal approach while drastically reducing computational cost, making it suitable for processing long video sequences in large video databases.

show abstract

Semantic mosaic for indexing and compressing instructional videos

Cited by 7 publications

References 14 publications

Visual Summarization of Lecture Video Segments for Enhanced Navigation

Visual Summarization of Lecture Video Segments for Enhanced Navigation

A two-level queueing system for interactive browsing and searching of video content

Computational approaches to temporal sampling of video sequences

Contact Info

Product

Resources

About