A real-time multimedia composition layer

Abstract

We introduce a light-weight multimedia composition layer which is situated on top of existing music, sound, and graphics frameworks and libraries. The layer provides applications with a programming interface for multimedia composition, automating real-time media processing and synchronisation. Our work facilitates the mapping of media frameworks to a unified processing graph. We present a graph segmentation algorithm that solves the problem of communication between threads of multiple frameworks while providing global consistency and without affecting processing performance. In addition, support for logical and physical time is included and enables framework-independent realisation of complex multimedia designs. The paper discusses the layer's architecture in detail and shows how multimedia applications can efficiently exploit its capabilities.

Integrating multicore awareness functions into distribution middleware for improving performance of distributed audio surveillance

“…Other Java-based contributions such as [26] and [27] have been directly based on the contributed Linux audio drivers. Other media-centric approaches are also available in the literature such as [28] that oers a framework for functional composition of dierent media streams.…”

A Synchronous Virtual Machine for Multimedia Presentations

“…The rationale behind the parachute policy is to avoid interrupting the processing flow, which is usually what is desirable in real-time operation [120]. While developing a Smix program, however, this policy is harmful since it tends to hide programming errors.…”