TranSmart: A Practical Interactive Machine Translation System

Huang, Guoping; Liu, Lemao; Wang, Xing; Wang, Longyue; Li, Huayang; Tu, Zhaopeng; Huang, Chunyue; Shi, Shuming

doi:10.48550/arxiv.2105.13072

Cited by 16 publications

(7 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As participation of LLM-based systems in our shared tasks increases, we anticipate integrating these benchmarks more comprehensively into our future evaluations to better measure and understand the evolution of LLM capabilities in linguistic context and discourse comprehension. Machine translation of web novels not only holds research value but also offers practical application prospects (Huang et al, 2021;. This shared task serves to spur competitive innovation and fosters the advancement of sophisticated machine translation systems capable of navigating the intricate nuances of literary works.…”

Section: Discussionmentioning

confidence: 99%

Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMs

Wang,

Tu,

et al. 2023

Proceedings of the Eighth Conference on Machine Translation

View full text Add to dashboard Cite

Translating literary works has perennially stood as an elusive dream in machine translation (MT), a journey steeped in intricate challenges. To foster progress in this domain, we hold a new shared task at WMT 2023, the first edition of the Discourse-Level Literary Translation. First, we (Tencent AI Lab and China Literature Ltd.) release a copyrighted and document-level Chinese-English web novel corpus. Furthermore, we put forth an industry-endorsed criteria to guide human evaluation process. This year, we totally received 14 submissions from 7 academia and industry teams. We employ both automatic and human evaluations to measure the performance of the submitted systems. The official ranking of the systems is based on the overall human judgments. In addition, our extensive analysis reveals a series of interesting findings on literary and discourse-aware MT. We release data, system outputs, and leaderboard at http://www2.statmt.org/wmt23/ literary-translation-task.html.

show abstract

Section: Discussionmentioning

confidence: 99%

Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMs

Wang,

Tu,

et al. 2023

Proceedings of the Eighth Conference on Machine Translation

View full text Add to dashboard Cite

show abstract

“…Their affirmation and encouragement have further solidified our commitment to the path of computational linguistics. This work is part of the GuoFeng AI (guofeng-ai@googlegroups.com) and TranSmart (Huang et al, 2021) projects.…”

Section: Acknowledgementmentioning

confidence: 99%

A Survey on Zero Pronoun Translation

Wang¹,

Liu²,

Xu³

et al. 2023

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Zero pronouns (ZPs) are frequently omitted in pro-drop languages (e.g. Chinese, Hungarian, and Hindi), but should be recalled in nonpro-drop languages (e.g. English). This phenomenon has been studied extensively in machine translation (MT), as it poses a significant challenge for MT systems due to the difficulty in determining the correct antecedent for the pronoun. This survey paper highlights the major works that have been undertaken in zero pronoun translation (ZPT) after the neural revolution so that researchers can recognize the current state and future directions of this field. We provide an organization of the literature based on evolution, dataset, method, and evaluation. In addition, we compare and analyze competing models and evaluation metrics on different benchmarks. We uncover a number of insightful findings such as: 1) ZPT is in line with the development trend of large language model; 2) data limitation causes learning bias in languages and domains; 3) performance improvements are often reported on single benchmarks, but advanced methods are still far from realworld use; 4) general-purpose metrics are not reliable on nuances and complexities of ZPT, emphasizing the necessity of targeted metrics; 5) apart from commonly-cited errors, ZPs will cause risks of gender bias.

show abstract

“…By comparing with commercial systems, we can gauge ChatGPT's performance relative to the best available MT technologies. We compare GPT-3.5/GPT-4 with three commercial translation products, including Google Translate, 6 DeepL Translate, 7 and Tencent TranSmart (Huang et al, 2021). 8 We employ both automatic (d-BLEU) and human evaluation (general/discourse-aware quality) as detailed in Section 2.2. perform LLM-based systems, except for the Q&A domain, which involves informal spoken language.…”

Section: Chatgpt Vs Commercial Systemsmentioning

confidence: 99%

Document-Level Machine Translation with Large Language Models

Wang,

Lyu,

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Large language models (LLMs) such as Chat-GPT can produce coherent, cohesive, relevant, and fluent answers for various natural language processing (NLP) tasks. Taking documentlevel machine translation (MT) as a testbed, this paper provides an in-depth evaluation of LLMs' ability on discourse modeling. The study focuses on three aspects: 1) Effects of Context-Aware Prompts, where we investigate the impact of different prompts on document-level translation quality and discourse phenomena; 2) Comparison of Translation Models, where we compare the translation performance of Chat-GPT with commercial MT systems and advanced document-level MT methods; 3) Analysis of Discourse Modelling Abilities, where we further probe discourse knowledge encoded in LLMs and shed light on impacts of training techniques on discourse modeling. By evaluating on a number of benchmarks, we surprisingly find that LLMs have demonstrated superior performance and show potential to become a new paradigm for document-level translation: 1) leveraging their powerful long-text modeling capabilities, GPT-3.5 and GPT-4 outperform commercial MT systems in terms of human evaluation; 1 2) GPT-4 demonstrates a stronger ability for probing linguistic knowledge than GPT-3.5. This work highlights the challenges and opportunities of LLMs for MT, which we hope can inspire the future design and evaluation of LLMs. 2 * Equal contribution. 1 The protocol employed in this work was approved by the Tencent Institutional Review Board (IRB).2 We release our data and annotations at https://github. com/longyuewangdcu/Document-MT-LLM.

show abstract

TranSmart: A Practical Interactive Machine Translation System

Cited by 16 publications

References 76 publications

Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMs

Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMs

A Survey on Zero Pronoun Translation

Document-Level Machine Translation with Large Language Models

Contact Info

Product

Resources

About