Adaptive Pitch-Shifting with Applications to Intonation Adjustment in a Cappella Recordings

Rosenzweig, Sebastian; Schwar, Simon; Driedger, Jonathan; Müller, Meinard

doi:10.23919/dafx51585.2021.9768268

Cited by 2 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The "Fixed Pitch-Shifting" technique is employed to ensure that the generated speech closely resembles the voice of the original speaker. Librosa provides the capability to detect the frequency of the audio and shift the pitch of the audio time series from one musical note to another (Rosenzweig et al, 2021). In the context of voice cloning, the mean frequency of the audio is determined, with the lower note con-4 sidered as F2 (87.31 Hz) and the higher note as G6 (98.00 Hz).…”

Section: Text To Audio Processingmentioning

confidence: 99%

TRAVID: An End-to-End Video Translation Framework

Adhikary,

Sugandhi,

Ghimire

et al. 2023

Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacifi

View full text Add to dashboard Cite

In today's globalized world, effective communication with people from diverse linguistic backgrounds has become increasingly crucial. While traditional methods of language translation, such as written text or voice-only translations, can accomplish the task, they often fail to capture the complete context and nuanced information conveyed through nonverbal cues like facial expressions and lip movements. In this paper, we present an end-to-end video translation system that not only translates spoken language but also synchronizes the translated speech with the lip movements of the speaker. Our system focuses on translating educational lectures in various Indian languages, and it is designed to be effective even in low-resource system settings. By incorporating lip movements that align with the target language and matching them with the speaker's voice using voice cloning techniques, our application offers an enhanced experience for students and users. This additional feature creates a more immersive and realistic learning environment, ultimately making the learning process more effective and engaging.

show abstract

Section: Text To Audio Processingmentioning

confidence: 99%