SHEF-Lite 2.0: Sparse Multi-task Gaussian Processes for Translation Quality Estimation

Beck, Daniel; Shah, Kashif; Specia, Lucia

doi:10.3115/v1/w14-3338

Cited by 16 publications

(18 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this task, the standard GP model outperformed the baseline, with the sparse GP model following very closely. These figures represent significant improvements compared to our submission to the same task in last year's shared task (Beck et al, 2013), where we were not able to beat the baseline. The main differences between last year's and this year's models are the use of additional datasets and a higher number of features (25 vs. 40).…”

Section: Official Results and Discussionmentioning

confidence: 78%

Proceedings of the Ninth Workshop on Statistical Machine Translation

Bojar

Buck

Federmann

et al. 2014

View full text Add to dashboard Cite

The focus of our workshop was to use parallel corpora for machine translation. Recent experimentation has shown that the performance of SMT systems varies greatly with the source language. In this workshop we encouraged researchers to investigate ways to improve the performance of SMT systems for diverse languages, including morphologically more complex languages, languages with partial free word order, and low-resource languages.Prior to the workshop, in addition to soliciting relevant papers for review and possible presentation, we conducted four shared tasks: a general translation task, a medical translation task, a quality estimation task, and a task to test automatic evaluation metrics. The medical translation task was introduced this year to address the important issue of domain adaptation within SMT. The results of the shared tasks were announced at the workshop, and these proceedings also include an overview paper for the shared tasks that summarizes the results, as well as provides information about the data used and any procedures that were followed in conducting or scoring the task. In addition, there are short papers from each participating team that describe their underlying system in greater detail.Like in previous years, we have received a far larger number of submission than we could accept for presentation. This year we have received 27 full paper submissions and 49 shared task submissions. In total WMT 2014 featured 12 full paper oral presentations and 49 shared task poster presentations.The invited talk was given by Alon Lavie (Carnegie Mellon University and Safaba Translation Solutions, Inc.) entitled "Machine Translation in Academia and in the Commercial World -a Contrastive Perspective".We would like to thank the members of the Program Committee for their timely reviews. We also would like to thank the participants of the shared task and all the other volunteers who helped with the evaluations. is a ranking of the systems that participated in its shared translation tasks, produced by aggregating pairwise sentencelevel comparisons collected from human judges. Over the past few years, there have been a number of tweaks to the aggregation formula in attempts to address issues arising from the inherent ambiguity and subjectivity of the task, as well as weaknesses in the proposed models and the manner of model selection.We continue this line of work by adapting the TrueSkill TM algorithm -an online approach for modeling the relative skills of players in ongoing competitions, such as Microsoft's Xbox Live -to the human evaluation of machine translation output. Our experimental results show that TrueSkill outperforms other recently proposed models on accuracy, and also can significantly reduce the number of pairwise annotations that need to be collected by sampling non-uniformly from the space of system competitions. IntroductionThe Workshop on Statistical Machine Translation (WMT) has long been a central event in the machine translation (MT) community for the evaluation of MT output. It hosts...

show abstract

Section: Official Results and Discussionmentioning

confidence: 78%

Proceedings of the Ninth Workshop on Statistical Machine Translation

Bojar

Buck

Federmann

et al. 2014

View full text Add to dashboard Cite

show abstract

“…Future work could use our proposed model to detect heavy sentences that needs such pre-processing. Our findings can also inspire informative features for sentence quality estimation, in which the task is to predict the sentence-level fluency (Beck et al, 2014). We have shown that heavy Chinese sentences are likely to lead to hard to read, disfluent sentences in English.…”

Section: Discussionmentioning

confidence: 99%

Detecting Content-Heavy Sentences: A Cross-Language Case Study

Nenkova

2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

The information conveyed by some sentences would be more easily understood by a reader if it were expressed in multiple sentences. We call such sentences content heavy: these are possibly grammatical but difficult to comprehend, cumbersome sentences. In this paper we introduce the task of detecting content-heavy sentences in cross-lingual context. Specifically we develop methods to identify sentences in Chinese for which English speakers would prefer translations consisting of more than one sentence. We base our analysis and definitions on evidence from multiple human translations and reader preferences on flow and understandability. We show that machine translation quality when translating content heavy sentences is markedly worse than overall quality and that this type of sentence are fairly common in Chinese news. We demonstrate that sentence length and punctuation usage in Chinese are not sufficient clues for accurately detecting heavy sentences and present a richer classification model that accurately identifies these sentences.

show abstract

“…GP regression models were recently successfully employed for post-editing time and HTER 2 prediction (Beck et al, 2013). Both used RBF kernels as the covariance function so a natural extension is to apply the structured kernels of Section 3.1.…”

Section: Quality Estimationmentioning

confidence: 99%

Bayesian Kernel Methods for Natural Language Processing

Beck¹

2014

Proceedings of the ACL 2014 Student Research Workshop

Self Cite

View full text Add to dashboard Cite

Kernel methods are heavily used in Natural Language Processing (NLP). Frequentist approaches like Support Vector Machines are the state-of-the-art in many tasks. However, these approaches lack efficient procedures for model selection, which hinders the usage of more advanced kernels. In this work, we propose the use of a Bayesian approach for kernel methods, Gaussian Processes, which allow easy model fitting even for complex kernel combinations. Our goal is to employ this approach to improve results in a number of regression and classification tasks in NLP.

show abstract

SHEF-Lite 2.0: Sparse Multi-task Gaussian Processes for Translation Quality Estimation

Cited by 16 publications

References 8 publications

Proceedings of the Ninth Workshop on Statistical Machine Translation

Proceedings of the Ninth Workshop on Statistical Machine Translation

Detecting Content-Heavy Sentences: A Cross-Language Case Study

Bayesian Kernel Methods for Natural Language Processing

Contact Info

Product

Resources

About