Incorporating feature hierarchy and boosting to achieve more effective classifier training and concept-oriented video summarization and skimming

Abstract

For online medical education purposes, we have developed a novel scheme to incorporate the results of semantic video classification to select the most representative video shots for generating concept-oriented summarization and skimming of surgery education videos. First, salient objects are used as the video patterns for feature extraction to achieve a good representation of the intermediate video semantics. The salient objects are defined as the salient video compounds that can be used to characterize the most significant perceptual properties of the corresponding real world physical objects in a video, and thus the appearances of such salient objects can be used to predict the appearances of the relevant semantic video concepts in a specific video domain. Second, a novel multi-modal boosting algorithm is developed to achieve more reliable video classifier training by incorporating feature hierarchy and boosting to dramatically reduce both the training cost and the size of training samples, thus it can significantly speed up SVM (support vector machine) classifier training. In addition, the unlabeled samples are integrated to reduce the human efforts on labeling large amount of training samples. Finally, the results of semantic video classification are incorporated to enable concept-oriented video summarization and skimming. Experimental results in a specific domain of surgery education videos are provided.

Proceedings of the 16th World Conference on Mobile and Contextual Learning

“…Automated summarization is a hot research topic with lots of work done towards characterizing general videos where edition cuts, different camera angles, recording locations and multiple characters can clearly mark shot boundaries greatly simplifying scene's feature extraction (Smith and Kanade 1998). Summarizing documentary educational videos (Luo, et al 2008, Song et al 2010) shares many aspects of general videos' summarization techniques with visually recognizable scenes. As for slides presentationbased lectures they can be summarized using already structured source files (Mukhopadhyay and Smith 1999).…”

Advanced Mobile Lecture Viewing

“…Automated summarization is a hot research topic (Money & Agius, 2008) and lots of work has been done towards characterizing general videos where edition cuts, different camera angles, recording locations and multiple individuals clearly mark shot boundaries greatly simplifying automated scene's feature extraction (Ciocca & Schettini, 2006a;Smith & Kanade, 1998). Summarizing documentary educational videos (Luo, Gao, Xue, Peng, & Fan, 2008;Song, Marchionini, & Oh, 2010) shares many aspects of general videos' summarization techniques with visually recognizable passages and cinematography elements. As for digital slides-based presentations and lectures they are commonly summarized based on their already structured source files in PowerPoint or PDF file formats (Mittal, Pagalthivarthi, & Altman, 2006;Mukhopadhyay & Smith, 1999;Repp & Meinel, 2006).…”