Joint Translation and Unit Conversion for End-to-end Localization

Dinu, Georgiana; Mathur, Prashant; Federico, Marcello; Lauly, Stanislas; Al-Onaizan, Yaser

doi:10.18653/v1/2020.iwslt-1.32

Cited by 1 publication

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…OpenSubtitles: OpenSubtitles has been used extensively, including by [Wang, 2017, Sjöblom et al, 2018, Zilio et al, 2018, Gordon and Duh, 2020, Krišlauks and Pinnis, 2020. Papers that train models on datasets that include the OpenSubstitles subset of the Pile include Luo et al [2021], Askell et al [2021] DM Mathematics: DM Mathematics has been used extensively, including by [Cho et al, 2019, Qi and Wu, 2019, Talmor et al, 2020, Dinu et al, 2020, Firestone, 2020. BookCorpus2: The BookCorpus dataset that BookCorpus2 is based on has been used extensively, including by [Karpathy and Fei-Fei, 2015, Reed et al, 2016, Ba et al, 2016, Devlin et al, 2018.…”

Section: Motivation For Dataset Creationmentioning

confidence: 99%

Datasheet for the Pile

Biderman¹,

Bicheno²,

Gao³

2022

Preprint

View full text Add to dashboard Cite

This datasheet describes the Pile, a 825 GiB dataset of human-authored text compiled by EleutherAI for use in large-scale language modeling. The Pile is comprised of 22 different text sources, ranging from original scrapes done for this project, to text data made available by the data owners, to third-party scrapes available online. Background on the PileThe Pile is a massive text corpus created by EleutherAI for large-scale language modeling efforts. It is comprised of textual data from 22 sources (see below) and can be downloaded from the official website as well as from a community mirror. Each source dataset is at its core a textual work, and any non-textual data (including metadata) has been removed. While still preserving their internal order, the documents from all the sources have been randomly shuffled. For further information on the Pile, see Gao et al. [2020].This document is not intended to be -and should not be used as -a substitute for a datasheet for the original versions of the component datasets. While it is accurate for the text data that we derived from each component dataset, the original source dataset may have other properties. This document is intended to inform people interested in using the Pile for natural language processing. People interested in using the original datasets should contact the data owners for information about the properties of the original data.It is not always the case that the answer to the questions below are known with certainty. For example, while we have no reason to believe that personal identifying information (PII) is contained in most of the subsets of our dataset, it is always possible that someone wrote down PII in a document and uploaded it to arXiv. Due to the sheer scale of the data, it is impractical to systematically search through every text to validate that it is what it purports to be. We have endeavored to answer the questions below as best we can, and to be open and honest about the limitations of the accuracy of this document. Anyone who engages in research on or with the Pile is welcome to contact us to have their findings added to this document. Similarly, we welcome all comments, suggestions, or corrections. Datasets contained in the Pile:Pile-CC: The Pile-CC dataset is a sample from the Common Crawl WARCs that has been converted to text using jusText [Endrédy and Novák, 2013].

show abstract