Automatic Indexing of Lecture Presentations Using Unsupervised Learning of Presumed Discourse Markers

Abstract

A new method for automatic detection of section boundaries and extraction of key sentences from lecture audio archives is proposed. The method makes use of 'discourse markers' (DMs), which are characteristic expressions used in initial utterances of sections, together with pause and language model information. The DMs are derived in a totally unsupervised manner based on word statistics. An experimental evaluation using the Corpus of Spontaneous Japanese (CSJ) demonstrates that the proposed method provides better indexing of section boundaries compared with a simple baseline method using pause information only, and that it is robust against speech recognition errors. The method is also applied to extraction of key sentences that can index the section topics. The statistics of the presumed DMs are used to define the importance of sentences, which favors potentially section-initial ones. The measure is also combined with the conventional tf-idf measure based on content words. Experimental results confirm the effectiveness of using the DMs in combination with the keyword-based method. The paper also describes a statistical framework for transforming raw speech transcriptions into the document style for defining appropriate sentence units and improving readability.

TalkMiner

“…Haubold and Kender [10,11] focus on multi-speaker presentation videos and develop an enhanced multimodal segmentation that leverages the audio stream to detect speaker changes. The use of audio for segmentation has also been studied in [14] for lectures and in more general purpose video retrieval systems [13,23]. Our work also employs visual frame differencing as a baseline, which we extend with both spatial analysis and speaker appearance modeling to reduce the number of non-slide images in the final set of keyframes.…”

Intelligent Transcription System Based on Spontaneous Speech Processing

“…The hypotheses of sentence boundaries are verified using the N-gram model. The method is also formulated as a statistical machine translation framework [22] and is referred to as enhanced SLM.…”

Application of Information Retrieval Technologies to Presentation Slides

“…More relevant information can be extracted from the audio. In [15], the analysis of prosody and silences as well as the detection of Discourse Markers, i.e. expressions that typically introduce a new argument, are used to segment and index university lectures.…”