FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech

Conneau, Alexis; Ma, Min; Khanuja, Simran; Dalmia, Siddharth; Riesa, Jason; Rivera, Clara E.; Bapna, Ankur

doi:10.1109/slt54892.2023.10023141

Cited by 57 publications

(28 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1. Speech Recognition: We evaluated on the test set of SpeechStew ASR [24], VoxPopuli ASR [25], and FLEURS ASR [26]. The performance was computed in terms of word error rate (WER) using the JiWER implementation [27].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Microwave ablation versus laparoscopic resection as first‐line therapy for solitary 3–5‐cm HCC

et al. 2022

View full text Add to dashboard Cite

Background and Aims The study objective was to compare the effectiveness of microwave ablation (MWA) and laparoscopic liver resection (LLR) on solitary 3–5‐cm HCC over time. Approach and Results From 2008 to 2019, 1289 patients from 12 hospitals were enrolled in this retrospective study. Diagnosis of all lesions were based on histopathology. Propensity score matching was used to balance all baseline variables between the two groups in 2008–2019 (n = 335 in each group) and 2014–2019 (n = 257 in each group) cohorts, respectively. For cohort 2008–2019, during a median follow‐up of 35.8 months, there were no differences in overall survival (OS) between MWA and LLR (HR: 0.88, 95% CI 0.65–1.19, p = 0.420), and MWA was inferior to LLR regarding disease‐free survival (DFS) (HR 1.36, 95% CI 1.05–1.75, p = 0.017). For cohort 2014–2019, there was comparable OS (HR 0.85, 95% CI 0.56–1.30, p = 0.460) and approached statistical significance for DFS (HR 1.33, 95% CI 0.98–1.82, p = 0.071) between MWA and LLR. Subgroup analyses showed comparable OS in 3.1–4.0‐cm HCCs (HR 0.88, 95% CI 0.53–1.47, p = 0.630) and 4.1–5.0‐cm HCCs (HR 0.77, 95% CI 0.37–1.60, p = 0.483) between two modalities. For both cohorts, MWA shared comparable major complications (both p > 0.05), shorter hospitalization, and lower cost to LLR (all p < 0.001). Conclusions MWA might be a first‐line alternative to LLR for solitary 3–5‐cm HCC in selected patients with technical advances, especially for patients unsuitable for LLR.

show abstract

Section: Discussionmentioning

confidence: 99%

“…We present the results of ASR evaluation in Table 1 on an English corpus using SpeechStew [24], as well as on multilingual corpora using Voxpopuli [25] and FLEURS [26]. The instructions during evaluation are similar to the ones in training (e.g., "Recognize this speech in {lang}").…”

Section: Speech Recognitionmentioning

confidence: 99%

Microwave ablation versus laparoscopic resection as first‐line therapy for solitary 3–5‐cm HCC

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Experiments on MuST-C (Cattoni et al, 2021) reveal that our method is not only better than existing zero-shot models, by a large margin, but also surpasses supervised ones, achieving state-of-the-art results. On CoVoST (Wang et al, 2020b), ZEROSWOT outperforms the original version of the multimodal SEAMLESSM4T (Seamless-Communication, 2023a), while evaluations on the 88 target languages of FLEURS (Conneau et al, 2023) showcase the massively multilingual capacity of our method. ZEROSWOT is also vastly superior to comparable CTC-based cascade ST models, and while it is on par with cascades that utilize strong attention-based encoder-decoder ASR models, it ranks better in terms of efficiency.…”

Section: Introductionmentioning

confidence: 84%

SHAS: Approaching optimal Segmentation for End-to-End Speech Translation

Tsiamas¹,

Gállego²,

Fonollosa³

et al. 2022

Interspeech 2022

View full text Add to dashboard Cite

Data scarcity and the modality gap between the speech and text modalities are two major obstacles of end-to-end Speech Translation (ST) systems, thus hindering their performance. Prior work has attempted to mitigate these challenges by leveraging external MT data and optimizing distance metrics that bring closer the speech-text representations. However, achieving competitive results typically requires some ST data. For this reason, we introduce ZEROS-WOT, a method for zero-shot ST that bridges the modality gap without any paired ST data. Leveraging a novel CTC compression and Optimal Transport, we train a speech encoder using only ASR data, to align with the representation space of a massively multilingual MT model. The speech encoder seamlessly integrates with the MT model at inference, enabling direct translation from speech to text, across all languages supported by the MT model. Our experiments show that we can effectively close the modality gap without ST data, while our results on MUST-C and COVOST demonstrate our method's superiority over not only previous zero-shot models, but also supervised ones, achieving state-of-the-art results. 1

show abstract

“…All these evaluations for speech were performed on FLEURS test set (Conneau et al, 2023), a N -way parallel speech dataset in 102 languages built on top of the text FLORES-101 benchmark.…”

Section: Evaluationsmentioning

confidence: 99%

Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer

Duquenne¹,

Schwenk²,

Sagot³

2023

Interspeech 2023

View full text Add to dashboard Cite

We introduce SONAR, a new multilingual and multimodal fixed-size sentence embedding space. Our single text encoder, covering 200 languages, substantially outperforms existing sentence embeddings such as LASER3 and LabSE on the xsim and xsim++ multilingual similarity search tasks. Speech segments can be embedded in the same SONAR embedding space using language-specific speech encoders trained in a teacher-student setting on speech transcription data. Our encoders outperform existing speech encoders on similarity search tasks. We also provide a text decoder for 200 languages, which allows us to perform text-to-text and speech-to-text machine translation, including for zero-shot language and modality combinations. Our text-to-text results are competitive compared to the state-of-theart NLLB 1B model, despite the fixed-size bottleneck representation. Our zero-shot speech-to-text translation results compare favorably with strong supervised baselines such as Whisper.

show abstract

FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech

Cited by 57 publications

References 31 publications

Microwave ablation versus laparoscopic resection as first‐line therapy for solitary 3–5‐cm HCC

Microwave ablation versus laparoscopic resection as first‐line therapy for solitary 3–5‐cm HCC

SHAS: Approaching optimal Segmentation for End-to-End Speech Translation

Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer

Contact Info

Product

Resources

About