Mixed Stereo Audio Classification Using a Stereo-Input Mixed-to-Panned Level Feature

Abstract

Many past studies have been conducted on speech/music discrimination due to the potential applications for broadcast and other media; however, it remains possible to expand the experimental scope to include samples of speech with varying amounts of background music. This paper focuses on the development and evaluation of two measures of the ratio between speech energy and music energy: a reference measure called speech-to-music ratio (SMR), which is known objectively only prior to mixing, and a feature called the stereo-input mix-to-peripheral level feature (SIMPL), which is computed from the stereo mixed signal as an imprecise estimate of SMR. SIMPL is an objective signal measure calculated by taking advantage of broadcast mixing techniques in which vocals are typically placed at stereo center, unlike most instruments. Conversely, SMR is a hidden variable defined by the relationship between the powers of portions of audio attributed to speech and music. It is shown that SIMPL is predictive of SMR and can be combined with state-of-the-art features in order to improve performance. For evaluation, this new metric is applied in speech/music (binary) classification, speech/music/mixed (trinary) classification, and a new speech-to-music ratio estimation problem. Promising results are achieved, including 93.06% accuracy for trinary classification and 3.86 dB RMSE for estimation of the SMR.

A Review on Machine Learning for Audio Applications

“…Due to the potential uses for broadcast and other media, many previous studies on speech/music discrimination have been undertaken; however, it is still conceivable to broaden the experimental scope to include samples of speech with varied quantities of background music. A. Chen et.al in [7] have discussed the development and evaluation of two measurements of the ratio between speech and music energy are the subject of this paper: a feature called the stereo-input mix-to-peripheral level feature (SIMPL), which is computed from the stereo mixed signal as an approximate estimate of SMR, and a reference measure called speech-to-music ratio (SMR), which is known objectively only prior to mixing. SIMPL is an objective signal measure determined using broadcast mixing procedures, in which vocals, unlike most instruments, are often positioned at stereo centre.…”

“…Gaussian mixture models (GMM) are used in this work because they perform similarly to the best reported methods on similar regression and classification tasks and can be implemented using ordinary voice recognition software. The research carried out by A. Chen et.al in [7], established a simple audio categorization feature that approximates the ratio of energy present in recorded voice or speaking and instrumental portions based on their normal stereo mix placements. SIMPL demonstrated to be a valuable feature in speech/music discrimination applications, with 81.9 percent success rates for three-way classification and 97.9% for two-way classification.…”

“…This is done in [6] by implementing an optimized audio classification and segmentation algorithm by the use of the ensembled bagged trees. In [7], mixed stereo audio classification is doneusing a stereo-input mixed-to-panned level feature. The new measure called the speech to music ratio is also calculated here.…”