GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation

Yang, Jian; Yin, Yuwei; Yang, Liqun; Ma, Shuming; Huang, Haoyang; Zhang, Dongdong; Wei, Furu; Li, Zhoujun

doi:10.1109/taslp.2022.3221040

Cited by 12 publications

(1 citation statement)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Building upon that, proposed Viterbi decoding and further explored fuzzy alignment training, achieving the current state-of-the-art NAR model performance. Apart from text translation, the NAR model also demonstrated impressive performance in diverse areas such as speech-to-text translation (Xu et al, 2023), speech-to-speech translation (Fang et al, 2023) and text-to-speech synthesis (Ren et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

Non-autoregressive Streaming Transformer for Simultaneous Translation

Ma,

Zhang,

Guo

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Simultaneous machine translation (SiMT) models are trained to strike a balance between latency and translation quality. However, training these models to achieve high quality while maintaining low latency often leads to a tendency for aggressive anticipation. We argue that such issue stems from the autoregressive architecture upon which most existing SiMT models are built. To address those issues, we propose non-autoregressive streaming Transformer (NAST) which comprises a unidirectional encoder and a non-autoregressive decoder with intra-chunk parallelism. We enable NAST to generate the blank token or repetitive tokens to adjust its READ/WRITE strategy flexibly, and train it to maximize the nonmonotonic latent alignment with an alignmentbased latency loss. Experiments on various SiMT benchmarks demonstrate that NAST outperforms previous strong autoregressive SiMT baselines. Source code is publicly available at https://github.com/ictnlp/NAST.

show abstract

Section: Related Workmentioning

confidence: 99%