Treemaps to visualise and navigate speech audio

Abstract

Audio recordings are usually treated as one unbreakable and sequential document. Most interfaces only support basic audio navigation controls such as play, pause, forward, and rewind. However, by extracting meaningful information from audio, such as the spoken words and acoustic noise, we have created a Treemap-based interface which makes the task of finding the important information in audio simple. When applied to lecture audio, our interface allows students to easily consume lecture recordings by only listening to the parts they are interested in. A user study shows that our interface can successfully help users to find content in lecture recordings.

Extracting audio summaries to support effective spoken document search

“…Past research has also explored ways of visually presenting summaries for multimedia, such as lecture recordings (Abdulhamid & Marshall, ; Abdulhamid, ; Munteanu, Penn, Baecker, & Zhang ; Stark, Whittaker, & Hirschberg, ) and spoken content from the cultural domain (Ordelman, Heeren, Huijbregts, de Jong, & Hiemstra, ; Heeren & de Jong, ). Typically, these include access via a visual interface to a playback functionality that allows users to search and browse audio content.…”