Event Mining in Multimedia Streams

Abstract

| Events are real-world occurrences that unfold over space and time. Event mining from multimedia streams improves the access and reuse of large media collections, and it has been an active area of research with notable recent progress. This paper contains a survey on the problems and solutions in event mining, approached from three aspects: event description, event-modeling components, and current event mining systems. We present a general characterization of multimedia events, motivated by the maxim of five BW[s and one BH[ for reporting real-world events in journalism: when, where, who, what, why, and how. We discuss the causes for semantic variability in real-world descriptions, including multilevel event semantics, implicit semantics facets, and the influence of context. We discuss five main aspects of an event detection system. These aspects are: the variants of tasks and event definitions that constrain system design, the media capture setup that collectively define the available data and necessary domain assumptions, the feature extraction step that converts the captured data into perceptually significant numeric or symbolic forms, statistical models that map the feature representations to richer semantic descriptions, and applications that use event metadata to help in different information-seeking tasks. We review current event-mining systems in detail, grouping them by the problem formulations and approaches. The review includes detection of events and actions in one or more continuous sequences, events in edited video streams, unsupervised event discovery, events in a collection of media objects, and a discussion on ongoing benchmark activities. These problems span a wide range of multimedia domains such as surveillance, meetings, broadcast news, sports, documentary, and films, as well as personal and online media collections. We conclude this survey with a brief outlook on open research directions.

Digital Data Streams: Creating Value from the Real-Time Flow of Big Data

“…12 In rhetoric, 5W+H are the circumstances, the elements (moria peristaseos-particulars of circumstance) of an event, and in narrative they represent the basic grammar of a story. Computer scientists attempting to extract events from multimedia DDSs have revitalized the 5W+H model, 13 which is now at the heart of several provenance ontologies. 14 While the model can readily handle structured (e.g., a single numeric) or ill-structured (e.g., a text string) data, extracting value from illstructured data is challenging and has galvanized considerable research on text mining.…”

Ubiquitous Event Mining to Enhance Personal Health

“…Every device and application should identify the associated events and write them to a shared event ledger. For example, a life logging application would find out when you are having a meal and write it to the ledger along with the associated metadata, including location, time stamp, and relevant experiential information [8], such as the name of the dish and an image if available. Figure 2 illustrates how we find the causal relationships from the event ledger and add them to the knowledge graph.…”

On a Framework for Complex and ad hoc Event Management over Distributed Systems

“…especially when the considered system is distributed over heterogeneous networks ("There is the need of querying and event processing architectures, simple schemes for sensor networks are not designed for multimedia sensors" [6]). Among the most interesting new approaches to multimedia event management, it is worth noting the six-dimensional "5W+1H" approach, derived from the journalism [7], [8] and particularly related to the emerging field of multimedia event management.…”

“…In a centralized system, time can be described as a completely ordered sequence of points 7 , where instants correspond to readings of the system local clock. Let e 1 and e 2 be two events, detected respectively at the instants I occ (e 1 ) and I occ (e 2 ).…”