A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU

Chen, Boxing; Cherry, Colin

doi:10.3115/v1/w14-3346

Cited by 166 publications

(112 citation statements)

References 5 publications

Supporting

Mentioning

106

Contrasting

Order By: Relevance

“…Fig. 7 shows the evaluation results for Meteor and BLEU with smoothing techniques proposed by Chen and Cherry [5]. These results also report the score as 1.0 for the exact similar answer sentences as noticed in Fig.…”

Section: Automatic Evaluationmentioning

confidence: 58%

“…In general the rest of the smoothing techniques correlate with others in an acceptable level, for instance, Table 9 A brief introduction to smoothing algorithms used with BLEU. The last three smoothing techniques are first proposed by Chen and Cherry [5]. These new ones are mostly formed by modifying the traditional ones shown in S1 to S4.…”

Section: Automatic Evaluationmentioning

confidence: 99%

“…This technique is a modification of the BLEU-S3 proposed by Chen and Cherry [5]. It changes the line 4 of the BLEU-S3 algorithm to invcnt = invcnt× K ln(len(T )) , where K is set empirically.…”

Section: Bleu-s5mentioning

confidence: 99%

“…This is another novel method introduced by Chen and Cherry [5]. It is based on that matched counts for similar n gram sizes should be similar.…”

Section: Bleu-s6mentioning

confidence: 99%

“…Although BLEU is more focused on corpus level analysis, however, we used it with smoothing techniques to work with sentence level analysis as described by Chen and Cherry [5]. Chen and Cherry [5] introduce four traditional smoothing techniques and three new techniques to use with BLEU. A brief description of these smoothing techniques are provided in Table 9.…”

Section: Automatic Evaluationmentioning

confidence: 99%

See 4 more Smart Citations

Utilizing typed dependency subtree patterns for answer sentence generation in question answering systems

2017

View full text Add to dashboard Cite

Question Answering over Linked Data (QALD) refers to the use of Linked Data by question answering systems and in recent times this has become increasingly popular as it opens up a massive Linked Data cloud which is a rich source of encoded knowledge. However, a major shortfall of current QALD systems is that they focus on presenting a single fact or factoid answer which is derived using SPARQL (SPARQL Protocol and RDF Query Language) queries. There is now an increased interest in development of human like systems which would be able to answer questions and even hold conversations by constructing sentences akin to humans. In this paper we introduce a new answer construction and presentation system, which utilizes the linguistic structure of the source question and the factoid answer to construct an answer sentence which closely emanates a human generated answer. We employ both Semantic Web technology and the linguistic structure to construct the answer sentences. The core of the research resides on extracting dependency subtree patterns from the questions and utilizing them in conjunction with the factoid answer to generate the answer sentence with a natural feel akin to an answer from a human when asked the question.We evaluated the system for both linguistic accuracy, and naturalness using human evaluation. These evaluation processes showed that the proposed approach is able to generate answer sentences which have R. Perera SECMS (D-58), Auckland University of Technology, Auckland 1010, New Zealand E-mail: rivindu.perera@aut.ac.nz P. Nand · A. Naeem SECMS (D-75), Auckland University of Technology, Auckland 1010, New Zealand linguistic accuracy and natural readability quotients of more than 70%. In addition, we also carried out a feasibility analysis on using automatic metrics for answer sentence evaluation. The results from this phase showed that the there is not a strong correlation between the results from automatic metric evaluation and the human ratings of the machine generated answers.

show abstract

Section: Automatic Evaluationmentioning

confidence: 58%

Section: Automatic Evaluationmentioning

confidence: 99%

Section: Bleu-s5mentioning

confidence: 99%

“…This is another novel method introduced by Chen and Cherry [5]. It is based on that matched counts for similar n gram sizes should be similar.…”

Section: Bleu-s6mentioning

confidence: 99%

Section: Automatic Evaluationmentioning

confidence: 99%

See 3 more Smart Citations

Utilizing typed dependency subtree patterns for answer sentence generation in question answering systems

2017

View full text Add to dashboard Cite

show abstract

Effective approaches to combining lexical and syntactical information for code summarization

Zhou

Fan

2020

Softw Pract Exp

View full text Add to dashboard Cite

Summary Natural language summaries of source codes are important during software development and maintenance. Recently, deep learning based models have achieved good performance on the task of automatic code summarization, which encode token sequence or abstract syntax tree (AST) of code with neural networks. However, there has been little work on the efficient combination of lexical and syntactical information of code for better summarization quality. In this paper, we propose two general and effective approaches to leveraging both types of information: a convolutional neural network that aims to better extract vector representation of AST node for downstream models; and a Switch Network that learns an adaptive weight vector to combine different code representations for summary generation. We integrate these approaches into a comprehensive code summarization model, which includes a sequential encoder for token sequence of code and a tree based encoder for its AST. We evaluate our model on a large Java dataset. The experimental results show that our model outperforms several state‐of‐the‐art models on various metrics, and the proposed approaches contribute a lot to the improvements.

show abstract

A Novel Linguistic-Aware Memory Structure for Enhancing the Response Generation

Wang

Yang

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU

Cited by 166 publications

References 5 publications

Utilizing typed dependency subtree patterns for answer sentence generation in question answering systems

Utilizing typed dependency subtree patterns for answer sentence generation in question answering systems

Effective approaches to combining lexical and syntactical information for code summarization

A Novel Linguistic-Aware Memory Structure for Enhancing the Response Generation

Contact Info

Product

Resources

About