Object-based audio for interactive football broadcast

Oldfield, Robert; Shirley, Ben; Spille, J.

doi:10.1007/s11042-013-1472-2

Cited by 17 publications

(10 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, Θ denotes the DNN parameters. Its example on a fully connected DNN is described later (after (16)). The target and observation source are assumed to be vectorized for all frequency bins as…”

Section: B Maximum-likelihood-based Dnn Training For T-f Mask Estimamentioning

confidence: 99%

“…First, the i-th observation utterance X (i) is simulated by (1) using a randomly selected target-source file and a noise source with equal frame size from the training dataset. Next, the T-F mask G(x (i) τ ) and variance σ(x (i) τ ) are estimated by (11)- (16). Then, to simulate the k-th output signalŜ (i,k) , the temporary output signalS (i,k) ω,τ is sampled from the complex Gaussian distribution using a pseudo random number generator, such as the Mersenne-Twister [44], as…”

Section: Training Proceduresmentioning

confidence: 99%

See 1 more Smart Citation

DNN-Based Source Enhancement to Increase Objective Sound Quality Assessment Score

Koizumi

Niwa

Hioka

et al. 2018

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

We propose a training method for deep neural network (DNN)-based source enhancement to increase objective sound quality assessment (OSQA) scores such as the perceptual evaluation of speech quality (PESQ). In many conventional studies, DNNs have been used as a mapping function to estimate time-frequency masks and trained to minimize an analytically tractable objective function such as the mean squared error (MSE). Since OSQA scores have been used widely for soundquality evaluation, constructing DNNs to increase OSQA scores would be better than using the minimum-MSE to create highquality output signals. However, since most OSQA scores are not analytically tractable, i.e., they are black boxes, the gradient of the objective function cannot be calculated by simply applying back-propagation. To calculate the gradient of the OSQA-based objective function, we formulated a DNN optimization scheme on the basis of black-box optimization, which is used for training a computer that plays a game. For a black-box-optimization scheme, we adopt the policy gradient method for calculating the gradient on the basis of a sampling algorithm. To simulate output signals using the sampling algorithm, DNNs are used to estimate the probability-density function of the output signals that maximize OSQA scores. The OSQA scores are calculated from the simulated output signals, and the DNNs are trained to increase the probability of generating the simulated output signals that achieve high OSQA scores. Through several experiments, we found that OSQA scores significantly increased by applying the proposed method, even though the MSE was not minimized.

show abstract

Section: B Maximum-likelihood-based Dnn Training For T-f Mask Estimamentioning

confidence: 99%

Section: Training Proceduresmentioning

confidence: 99%

DNN-Based Source Enhancement to Increase Objective Sound Quality Assessment Score

Koizumi

Niwa

Hioka

et al. 2018

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…However, in future content production, it might be possible to capture media assets directly into an object-based form. This approach has been applied for experimental live sports broadcasting [20], [21], but is not yet commonplace for audio production. The proposed system offers new opportunities for object-based audio capture based on performer tracking (to obtain metadata) and the application of BSS and beamforming techniques to spatial audio capture (to acquire separated object audio).…”

Section: B Component-based Designmentioning

confidence: 99%

An Audio-Visual System for Object-Based Audio: From Recording to Listening

Coleman

Franck

Francombe

et al. 2018

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Object-based audio is an emerging representation for audio content, where content is represented in a reproductionformat-agnostic way and thus produced once for consumption on many different kinds of devices. This affords new opportunities for immersive, personalized, and interactive listening experiences. This article introduces an end-to-end object-based spatial audio pipeline, from sound recording to listening. A high-level system architecture is proposed, which includes novel audiovisual interfaces to support object-based capture and listenertracked rendering, and incorporates a proposed component for objectification, i.e., recording content directly into an object-based form. Text-based and extensible metadata enable communication between the system components. An open architecture for object rendering is also proposed. The system's capabilities are evaluated in two parts. First, listener-tracked reproduction of metadata automatically estimated from two moving talkers is evaluated using an objective binaural localization model. Second, object-based scene capture with audio extracted using blind source separation (to remix between two talkers) and beamforming (to remix a recording of a jazz group), is evaluated with perceptually-motivated objective and subjective experiments. These experiments demonstrate that the novel components of the system add capabilities beyond the state of the art. Finally, we discuss challenges and future perspectives for object-based audio workflows.

show abstract

“…In current TV broadcasts these discrete audio object categories are not available at any point in the broadcast production chain and therefore the end user has no control over the relative levels of these sounds. In order to provide these three sound sources as independent and controllable entities some considerable development had to take place in the acquisition and production techniques used to capture a complex sound scene such as that found at a live sports event [64]. Currently the key objectives for audio in football coverage are twofold; picking up sounds on the pitch as clearly as possible during the game and utilizing the 5.1 surround sound capability to give the viewer a sense of immersion and of "being there."…”

Section: Clean Audio For Live Sports Coveragementioning

confidence: 99%