M4note: a multimodal tool for multimedia annotations

Abstract

This paper discusses an approach to the problem of annotating multimedia content. Our approach provides annotation as metadata for indexing, retrieval and semantic processing as well as content enrichment. We use an underlying model for structured multimedia descriptions and annotations, allowing the establishment of spatial, temporal and linking relationships. We discuss aspects related with documents and annotations used to guide the design of an application that allows annotations to be made with pen-based interaction with Tablet PCs. As a result, a video stream can be annotated at the same time that it is captured. Moreover, the annotation can be edited, extended or played back synchronously afterwards.

An enhanced content selection mechanism for personalization of video news programmes

“…Usually, to provide a pleasant enrichment tool to the user, the interaction follows the natural interfaces concept, previously defined by Abowd and Mynatt [1] as a subarea of ubiquitous computing [47]. Thus, there is a variety of work [7][8][9]14] which simulates the action of handwriting into a piece of content, using, for instance, a pen-based device like a tablet augmenter or a touchscreen mobile phone. Regularly, strokes (handwritten electronic ink marks) produced by the user are stored locally, and later, they can be used to create personalized multimedia presentations to be shared with friends.…”

FaericWorld: Browsing Multimedia Events Through Static Documents and Links

“…M4Note is a multimodal system for video recording that automatically computes low-level features and that allows users to easily add manual annotations thanks to electronic ink and speech recognition [11]. In the same way, Campanella et al described a system that visualizes features automatically extracted from videos, in order to support users in defining clusters of shots [5].…”

Context-aware support in structured documents for interactive-TV

“…M4Note [28,29] is a multimedia multimodal annotation tool that allows electronic ink and voice annotations on frames extracted from a video stream presented on a Tablet PCs -when a camera is used, as in Fig. 3, a video stream can be annotated at the same time that it is recorded.…”

“…Although the first approach, annotation by metadata association, has the potential to allow matching and sharing of the exact meaning of descriptions, its use can be quite arduous. However, when automatic metadata extraction and personalized tag hierarchies can be applied, it is possible to find a compromise between accuracy and effort [28,29]. The second approach, annotation by content enrichment, may be more appealing from the user's point of view since it does not require previous knowledge of tag hierarchies and is a natural way of annotating considering annotations made by voice or ink marking.…”

“…Descriptions relative to whole program and descriptions of each component (the media objects) are necessary so that applications may match the users' interests [5,7,16,26,28,29]. Contextual information can help this task, for example supporting the identification of who is in a scene or where the scene occurs when a user clicks on the image (see Sect.…”