Low Bit-Rate Speech Codec Based on a Long-Term Harmonic Plus Noise Model

Ali, Faten Ben; Djaziri-Larbi, Sonia; Girin, Laurent

doi:10.17743/jaes.2016.0028

“…Yet these approaches are limited to one-step frame prediction. A few other "unconventional" studies (Atal, 1983;Farvardin & Laroia, 1989;Mudugamuwa & Bradley, 1998;Dusan, Flanagan, Karve, & Balaraman, 2007;Girin, Firouzmand, & Marchand, 2007;Girin, 2010;Ben Ali, Djaziri-Larbi, & Girin, 2016) have proposed "long-term" speech coders, which aim at exploiting speech signal redundancy and predictability over larger time spans, typically in the range of a few hundreds of milliseconds. 1 However, these methods actually implement a joint coding of several short-term frames (basically, by using trajectory models or projections) but do not apply any explicit prediction of a frame given past frames.…”

Section: Predictions In Speech Coding Systemsmentioning

confidence: 99%

Evaluating the Potential Gain of Auditory and Audiovisual Speech-Predictive Coding Using Deep Learning

Hueber

¹

,

Tatulli

²

,

Girin

³

et al. 2020

Self Cite

View full text Add to dashboard Cite

Sensory processing is increasingly conceived in a predictive framework in which neurons would constantly process the error signal resulting from the comparison of expected and observed stimuli. Surprisingly, few data exist on the accuracy of predictions that can be computed in real sensory scenes. Here, we focus on the sensory processing of auditory and audiovisual speech. We propose a set of computational models based on artificial neural networks (mixing deep feedforward and convolutional networks), which are trained to predict future audio observations from present and past audio or audiovisual observations (i.e., including lip movements). Those predictions exploit purely local phonetic regularities with no explicit call to higher linguistic levels. Experiments are conducted on the multispeaker LibriSpeech audio speech database (around 100 hours) and on the NTCD-TIMIT audiovisual speech database (around 7 hours). They appear to be efficient in a short temporal range (25–50 ms), predicting 50% to 75% of the variance of the incoming stimulus, which could result in potentially saving up to three-quarters of the processing power. Then they quickly decrease and almost vanish after 250 ms. Adding information on the lips slightly improves predictions, with a 5% to 10% increase in explained variance. Interestingly the visual gain vanishes more slowly, and the gain is maximum for a delay of 75 ms between image and predicted sound.

show abstract

“…1 Still, these 76 approaches are limited to 1-step frame prediction, because of constraints on latency in 77 telecommunications. Some other studies [28][29][30][31][32][33][34] have proposed "middle-term" speech 78 coders, which aim at exploiting the speech signal redundancy and predictability over 79 larger time spans, typically in the range of a few hundreds of ms. However, these 80 methods actually implement a joint coding of several short-term frames (basically, by 81 using trajectory models or projections), but do not apply any explicit prediction of a 82 frame given past frames.…”

mentioning

confidence: 99%

How predictive can be predictions in the neurocognitive processing of auditory and audiovisual speech? A deep learning study

Hueber

¹

,

Tatulli

²

,

Girin

³

et al. 2018

Preprint

Self Cite

0

View full text Add to dashboard Cite

Sensory processing is increasingly conceived in a predictive framework in which neurons would constantly process the error signal resulting from the comparison of expected and observed stimuli. Surprisingly, few data exist on the amount of predictions that can be computed in real sensory scenes. Here, we focus on the sensory processing of auditory and audiovisual speech. We propose a set of computational models based on artificial neural networks (mixing deep feed-forward and convolutional networks) which are trained to predict future audio observations from 25 ms to 250 ms past audio or audiovisual observations (i.e. including lip movements). Experiments are conducted on the multispeaker NTCD-TIMIT audiovisual speech database. Predictions are efficient in a short temporal range (25-50 ms), predicting 40 to 60 % of the variance of the incoming stimulus, which could result in potentially saving up to 2/3 of the processing power. Then they quickly decrease to vanish after 100 ms. Adding information on the lips slightly improves predictions, with a 5 to 10 % increase in explained variance. Interestingly the visual gain vanishes more slowly, and the gain is maximum for a delay of 75 ms between image and predicted sound. 5 experimental developments, the predictive brain has been mathematically encapsulated 6 by Friston and colleagues into a powerful framework based on Bayesian modeling [3], 7 associating such concepts as perceptual inference [4], reinforcement learning [5] and 8 optimal control [6]. In this framework, it has been proposed that the minimization of 9 free energy -a concept coming from thermodynamics-could provide a general principle 10 associating perception and action in interaction with the environment in a coherent 11 predictive process [7,8]. A number of recent neurophysiological studies confirm the 12 accuracy of the predictive coding paradigm for analyzing sensory processing in the 13 human brain (e.g. [9]). 14 Actually, predictive coding is a general methodological paradigm in information 15 processing that consists in analyzing the local regularities in an input data stream in 16 1/25 order to extract the predictable part of these input data. The information processing 17 system can then focus on the difference between input data and their prediction. In a 18 very general manner, whatever the processing system, there are two main advantages to 19 processing the difference signal over directly processing the input signal. First, if the 20 prediction is efficient, the difference signal is generally of (much) lower energy than the 21 original signal, which leads to energy consumption saving in subsequent processes and 22 resource saving for representing the signal with a given accuracy (e.g. bitrate saving in 23 an audio or a video coder). In short, this reduces the "cost" of information processing. 24 Second, there is a concentration of novelty / unpredictable information in the difference 25 signal, which is exploitable for, e.g., the detection of new events. Because of these 26 advantages, predi...

show abstract

A long term harmonic plus noise model for narrow-band speech coding at very low bit-rates

Ali

¹

,

Djaziri-Larbi

²

2017

2017 40th International Conference on Telecommunications and Signal Processing (TSP)

1

0

View full text Add to dashboard Cite

Low Bit-Rate Speech Codec Based on a Long-Term Harmonic Plus Noise Model

Cited by 3 publications

References 31 publications

Evaluating the Potential Gain of Auditory and Audiovisual Speech-Predictive Coding Using Deep Learning

Evaluating the Potential Gain of Auditory and Audiovisual Speech-Predictive Coding Using Deep Learning

How predictive can be predictions in the neurocognitive processing of auditory and audiovisual speech? A deep learning study

A long term harmonic plus noise model for narrow-band speech coding at very low bit-rates

Contact Info

Product

Resources

About