CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022

Abstract

In this paper, we describe our submission to the Simultaneous Speech Translation at IWSLT 2022. We explore strategies to utilize an offline model in a simultaneous setting without the need to modify the original model. In our experiments, we show that our onlinization algorithm is almost on par with the offline setting while being 3× faster than offline in terms of latency on the test set. We also show that the onlinized offline model outperforms the best IWSLT2021 simultaneous system in medium and high latency regimes and is almost on par in the low latency regime. We make our system publicly available. 1

Incremental Blockwise Beam Search for Simultaneous Speech Translation with Controllable Quality-Latency Tradeoff

“…Specifically, [6] proposed a hold-n policy that removes the last n tokens from the model output, and local agreement that finds the longest common prefix of outputs obtained for two consecutive input contexts. Moreover, [7] showed that varying the chunk size can also be effectively applied along these policies.…”

“…The advantage of the standard beam search is that the model can generate a complete translation for current speech input. However, it is also prone to overgeneration and low-quality translations toward the end of the context [7]. In ASR, the standard beam search with attentional models shown poor length generalization [27].…”

“…All models are evaluated using Simuleval [19] toolkit. For the translation quality, we report detokenized case-sensitive BLEU [37], and for the latency, we report length-aware average lagging (LAAL) [7,38]. In all our experiments, we use beam search with size 6.…”

“…However, wait-k cannot directly use beam search, and its direct application to speech input is also complicated [5]. Alternatively, we can leave the model to generate the whole translation for the current context and heuristically decide which portion of the translation is reliable [6,7]. However, relying on attention can lead to over-generation and low-quality translation [7,8].…”

Long-form Simultaneous Speech Translation: Thesis Proposal

“…Therefore, we proposed an improved version of the AL metric, which was later independently proposed under name length-adaptive average lagging (LAAL; Papi et al, 2022). To remedy the over-generation problem, we proposed an improved version of the beam search algorithm in Polák et al (2023b). While this led to significant improvements in the quality-latency tradeoff, the decoding still relied on label-synchronous decoding.…”