Combining Automatic Speaker Verification and Prosody Analysis for Synthetic Speech Detection

Attorresi, Luigi; Salvi, Davide; Borrelli, Clara; Bestagini, Paolo; Tubaro, Stefano

doi:10.1007/978-3-031-37742-6_21

Cited by 8 publications

(2 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors of [42] exploit the lack of emotional content in synthetic voices generated via TTS techniques to recognize them. Finally, in [43] ASV and prosody features are combined to perform synthetic speech detection.…”

Section: Speech Deepfake Detection Methodsmentioning

confidence: 99%

“…Although silence is a fundamental component of speech, this is often overlooked in data generation, leading to biased tracks that are easy to discriminate [67]. This is a common problem, especially when dealing with TTS algorithms, where the prosodic component is less present [43] and the duration of the silences is shorter. Table 3 shows the silence durations of our tracks for both the original and the DTW cases.…”

Section: A Timit-tts Statisticsmentioning

confidence: 99%

See 1 more Smart Citation

TIMIT-TTS: A Text-to-Speech Dataset for Multimodal Synthetic Media Detection

et al. 2023

Self Cite

View full text Add to dashboard Cite

With the rapid development of deep learning techniques, the generation and counterfeiting of multimedia material has become increasingly simple. Current technology enables the creation of videos where both the visual and audio contents are falsified. While the multimedia forensics community has begun to address this threat by developing fake media detectors. However, the vast majority existing forensic techniques only analyze one modality at a time. This is an important limitation when authenticating manipulated videos, because sophisticated forgeries may be difficult to detect without exploiting cross-modal inconsistencies (e.g., across the audio and visual tracks). One important reason for the lack of multimodal detectors is a similar lack of research datasets containing multimodal forgeries. Existing datasets typically contain only one falsified modality, such as deepfaked videos with authentic audio tracks, or synthetic audio with no associated video. Currently, datasets are needed that can be used to develop, train, and test these forensic algorithms. In this paper, we propose a new audio-visual deepfake dataset containing multimodal video forgeries. We present a general pipeline for synthesizing deepfake speech content from a given video, facilitating the creation of counterfeit multimodal material. The proposed method uses Text-to-Speech (TTS) and Dynamic Time Warping (DTW) techniques to achieve realistic speech tracks. We use this pipeline to generate and release TIMIT-TTS, a synthetic speech dataset containing the most cutting-edge methods in the TTS field. This can be used as a standalone audio dataset, or combined with DeepfakeTIMIT and VidTIMIT video datasets to perform multimodal research. Finally, we present numerous experiments to benchmark the proposed dataset in both monomodal (i.e., audio) and multimodal (i.e., audio and video) conditions. This highlights the need for multimodal forensic detectors and more multimodal deepfake data.

show abstract

Section: Speech Deepfake Detection Methodsmentioning

confidence: 99%

Section: A Timit-tts Statisticsmentioning

confidence: 99%