NeurST: Neural Speech Translation Toolkit

Abstract

NeurST is an open-source toolkit for neural speech translation. The toolkit mainly focuses on end-to-end speech translation, which is easy to use, modify, and extend to advanced speech translation research and products. NeurST aims at facilitating the speech translation research for NLP researchers and building reliable benchmarks for this field. It provides step-by-step recipes for feature extraction, data preprocessing, distributed training, and evaluation. In this paper, we will introduce the framework design of NeurST and show experimental results for different benchmark datasets, which can be regarded as reliable baselines for future research. The toolkit is publicly available at https://github. com/bytedance/neurst and we will continuously update the performance of NeurST with other counterparts and studies at https: //st-benchmark.github.io/.

The Volctrans Neural Speech Translation System for IWSLT 2021

“…We conduct all our experiments using NeurST (Zhao et al, 2020) and report results for the submitted speech translation tasks in this section.…”

“…The results of our end-to-end solutions are presented in line 8-20, where line 8 is a benchmark model (Zhao et al, 2020) trained on the MuST-C dataset only. With the growth of model capacity (256d→768d) and data augmentation, we obtain 6.2 BLEU improvement on the tst-COMMON over the benchmark (line 8).…”

GigaST: A 10,000-hour Pseudo Speech Translation Corpus

“…In this section, we conduct ST experiments with speech transformer models and SSL-Transformer models. All models are implemented using NeurST [5].…”

“…In detail, we lowercase the English transcriptions, remove all punctuations and use SentencePiece 4 with a vocabulary of 15,000. For Chinese text, we first segment sentences by Jieba 5 , and then apply Byte-Pair Encoding (BPE) 6 [25] with 32,000 merge operations. German texts are first tokenized using Moses tokenizer, followed by BPE with 32,000 merge operations.…”

“…End-to-end speech translation (ST) directly translates the speech in the source language into sentences in the target language, without outputting the source language transcription [1]. With the success of attention-based models for speech and textrelated tasks, a typical and effective baseline model for speech translation tasks is speech-transformer [2,3,4,5], which has much of the same model structure as the commonly used MT model Transformer [6], except for the pre-processing downsampling module for speech signals.…”