Time-Scale Modification of Audio Signals Using Enhanced WSOLA With Management of Transients

Abstract-The continuing growth in the mobile phone arena, particularly in terms of device capabilities and ownership is having a transformational impact on media consumption. It is now possible to consider orchestrated multi-stream experiences delivered across many devices, rather than the playback of content from a single device. However, there are significant challenges in realising such a vision, particularly around the management of synchronicity between associated media streams. This is compounded by the heterogeneous nature of user devices, the networks upon which they operate, and the perceptions of users. This paper describes IMSync, an open inter-stream synchronisation framework that is QoE-aware. IMSync adopts efficient monitoring and control mechanisms, alongside a QoE perception model that has been derived from a series of subjective user experiments. Based on an observation of lag, IMSync is able to use this model of impact to determine an appropriate strategy to catch-up with playback whilst minimising the potential detrimental impacts on a users QoE. The impact model adopts a balanced approach: trading off the potential impact on QoE of initiating a re-synchronisation process compared with retaining the current levels of non-synchronicity, in order to maintain high levels of QoE. A series of experiments demonstrate the potential of the framework as a basis for enabling new, immersive media experiences.

show abstract

“…WSOLA [12], [38]). We currently use audio and video as the reference media to model the human perception of media synchronization.…”

Section: Discussionmentioning

confidence: 99%

Closing the Gap: Human Factors in Cross-Device Media Synchronization

Fawcett

Bird

et al. 2017

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

show abstract

“…In [13], the idea is to first identify the temporal positions of transients in the input signal by using a transient detector. Then, in the WSOLA process, the analysis hopsize is temporarily fixed to be equal to the synthesis hopsize whenever an analysis frame is located in a neighborhood of an identified transient.…”

Section: Tricks Of the Tradementioning

confidence: 99%

“…Percussive component In contrast to other transient preserving TSM procedures such as [13,22,24,29], the approach based on HPS has the advantage that an explicit detection and preservation of transients is not necessary. Figure 12 shows the same input signal x as in Figure 10.…”

Section: Harmonic Componentmentioning

confidence: 99%

A Review of Time-Scale Modification of Music Signals

Driedger

Müller

2016

Applied Sciences

View full text Add to dashboard Cite

Time-scale modification (TSM) is the task of speeding up or slowing down an audio signal's playback speed without changing its pitch. In digital music production, TSM has become an indispensable tool, which is nowadays integrated in a wide range of music production software. Music signals are diverse-they comprise harmonic, percussive, and transient components, among others. Because of this wide range of acoustic and musical characteristics, there is no single TSM method that can cope with all kinds of audio signals equally well. Our main objective is to foster a better understanding of the capabilities and limitations of TSM procedures. To this end, we review fundamental TSM methods, discuss typical challenges, and indicate potential solutions that combine different strategies. In particular, we discuss a fusion approach that involves recent techniques for harmonic-percussive separation along with time-domain and frequency-domain TSM procedures.

show abstract

“…Rhythm is changed by using the WSOLA algorithm [10]. This approach is similar to LP-PSOLA but periods are determined in real-time based on cross-correlation (3) and average magnitude difference (4) metrics.…”

Section: Speech Element Modificationmentioning

confidence: 99%

Improving Speech Synthesis Quality for Voices Created from an Audiobook Database

Chistikov

Zakharov²,

Talanov³

2014

Speech and Computer

View full text Add to dashboard Cite

Abstract. This paper describes an approach to improving synthesized speech quality for voices created by using an audiobook database. The data consist of a large amount of read speech by one speaker, which we matched with the corresponding book texts. The main problems with such a database are the following. First, the recordings were made at different times under different acoustic conditions, and the speaker reads the text with a variety of intonations and accents, which leads to very high voice parameter variability. Second, automatic techniques for sound file labeling make more errors due to the large variability of the database, especially as there can be mismatches between the text and the corresponding sound files. These problems dramatically affect speech synthesis quality, so a robust method for solving them is vital for voices created using audiobooks. The approach described in the paper is based on statistical models of voice parameters and special algorithms of speech element concatenation and modification. Listening tests show that it strongly improves synthesized speech quality.

show abstract

Time-Scale Modification of Audio Signals Using Enhanced WSOLA With Management of Transients

Cited by 38 publications

References 27 publications

Closing the Gap: Human Factors in Cross-Device Media Synchronization

Closing the Gap: Human Factors in Cross-Device Media Synchronization

A Review of Time-Scale Modification of Music Signals

Improving Speech Synthesis Quality for Voices Created from an Audiobook Database

Contact Info

Product

Resources

About