All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text

Clark, Elizabeth A.; August, Tal; Serrano, Sofia; Haduong, Nikita; Gururangan, Suchin; Smith, Noah A.

doi:10.48550/arxiv.2107.00061

Cited by 25 publications

(31 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6), using a linear regression model predicting trial-by-trial judgments as a function of categorical variables encoding sentence length (short, medium, long) and the source of the sentence (Wikipedia, GSN, MH, LSTM, or n-gram). First, we find that the naturalness of sentences from GSN declines by 14 points at longer sentence lengths, p < 0.001, while the naturalness of Wikipedia sentences is unaffected by length (in-7 See Clark et al (2021) for a discussion of the merits of phrasing the question in terms of naturalness instead of asking participants to judge whether it was produced by a human or machine. teraction term, p < 0.001), consistent with results reported by Ippolito et al (2020).…”

Section: Behavioral Resultsmentioning

confidence: 92%

Probing BERT's priors with serial reproduction chains

Yamakoshi¹,

Griffiths²,

Hawkins³

2022

Preprint

View full text Add to dashboard Cite

We can learn as much about language models from what they say as we learn from their performance on targeted benchmarks. Sampling is a promising bottom-up method for probing, but generating samples from successful models like BERT remains challenging. Taking inspiration from theories of iterated learning in cognitive science, we explore the use of serial reproduction chains to probe BERT's priors. Although the masked language modeling objective does not guarantee a consistent joint distribution, we observe that a unique and consistent estimator of the ground-truth joint distribution may be obtained by a GSN sampler, which randomly selects which word to mask and reconstruct on each step. We compare the lexical and syntactic statistics of sentences from the resulting prior distribution against those of the ground-truth corpus distribution and elicit a large empirical sample of naturalness judgments to investigate how, exactly, the model deviates from human speakers. Our findings suggest the need to move beyond top-down evaluation methods toward bottomup probing to capture the full richness of what has been learned about language.

show abstract

Section: Behavioral Resultsmentioning

confidence: 92%

Probing BERT's priors with serial reproduction chains

Yamakoshi¹,

Griffiths²,

Hawkins³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent years have brought considerable improvements in the language generation capabilities of neural language models (Lewis et al 2020;Ji et al 2020;Brown et al 2020;Holtzman et al 2020), allowing users of these systems to pass off their generations as human-produced (Ippolito et al 2020). These advances have raised dual-use concerns as to whether these tools could be used to generate text for malicious purposes (Radford et al 2019;Bommasani et al 2021), which humans would struggle to detect (Clark et al 2021).…”

Section: Synthetic Disinformation Generationmentioning

confidence: 99%

Synthetic Disinformation Attacks on Automated Fact Verification Systems

Bosselut

Manning

2022

AAAI

View full text Add to dashboard Cite

Automated fact-checking is a needed technology to curtail the spread of online misinformation. One current framework for such solutions proposes to verify claims by retrieving supporting or refuting evidence from related textual sources. However, the realistic use cases for fact-checkers will require verifying claims against evidence sources that could be affected by the same misinformation. Furthermore, the development of modern NLP tools that can produce coherent, fabricated content would allow malicious actors to systematically generate adversarial disinformation for fact-checkers. In this work, we explore the sensitivity of automated fact-checkers to synthetic adversarial evidence in two simulated settings: ADVERSARIAL ADDITION, where we fabricate documents and add them to the evidence repository available to the fact-checking system, and ADVERSARIAL MODIFICATION, where existing evidence source documents in the repository are automatically altered. Our study across multiple models on three benchmarks demonstrates that these systems suffer significant performance drops against these attacks. Finally, we discuss the growing threat of modern NLG systems as generators of disinformation in the context of the challenges they pose to automated fact-checkers.

show abstract

“…One of the latest tools, RoFT (Real or Fake Text) tool (Dugan et al, 2020), is used to evaluate human detection showing that the text generation models are capable to fool humans by one or two sentences. A recent research (Clark et al, 2021) shows that training humans on evaluation task for GPT-3 generated text only improves the overall accuracy upto 50%.Despite the interest in measuring the ability of humans to detect automatically generated text, not much research has been conducted to develop automatic tools to distinguish machine generated text from human written text.…”

Section: Human Detection Of Machine Generated Textmentioning

confidence: 99%

A Benchmark Corpus for the Detection of Automatically Generated Text in Academic Publications

Liyanage¹,

Buscaldi²,

Nazarenko³

2022

Preprint

View full text Add to dashboard Cite

Automatic text generation based on neural language models has achieved performance levels that make the generated text almost indistinguishable from those written by humans. Despite the value that text generation can have in various applications, it can also be employed for malicious tasks. The diffusion of such practices represent a threat to the quality of academic publishing. To address these problems, we propose in this paper two datasets comprised of artificially generated research content: a completely synthetic dataset and a partial text substitution dataset. In the first case, the content is completely generated by the GPT-2 model after a short prompt extracted from original papers. The partial or hybrid dataset is created by replacing several sentences of abstracts with sentences that are generated by the Arxiv-NLP model. We evaluate the quality of the datasets comparing the generated texts to aligned original texts using fluency metrics such as BLEU and ROUGE. The more natural the artificial texts seem, the more difficult they are to detect and the better is the benchmark. We also evaluate the difficulty of the task of distinguishing original from generated text by using state-of-the-art classification models.

show abstract

All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text

Cited by 25 publications

References 16 publications

Probing BERT's priors with serial reproduction chains

Probing BERT's priors with serial reproduction chains

Synthetic Disinformation Attacks on Automated Fact Verification Systems

A Benchmark Corpus for the Detection of Automatically Generated Text in Academic Publications

Contact Info

Product

Resources

About