Analyzing the unrestricted web: The finnish corpus of online registers

Skantsi, Valtteri; Laippala, Veronika

doi:10.1017/s0332586523000021

Cited by 5 publications

(7 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Removal of machine-translated and machine-generated content is a typical step in construction of web-based corpora. To this end, we trained a dedicated classifier using the FinCORE dataset (Skantsi & Laippala, 2023) discussed in greater detail in Section 3.4.4. This step removed the most material out of all the filtering steps, resulting in the removal of more than a billion tokens, as many low-quality noisy documents were identified in this step, in addition to genuine machine translated content.…”

Section: Removal Of Machine-translated Contentmentioning

confidence: 99%

“…Other linguistic studies which make use of the dependency syntax structures include a study on discourse connectives (Laippala, Kyröläinen, Kanerva, & Ginter, 2018) and on emoticons . The corpus also served as the source data for the work of Skantsi and Laippala (2023) on Finnish text register classification, used to provide the text register metadata described in Section 3.4.4.…”

Section: Linguistic Researchmentioning

confidence: 99%

“…Register (genre) information was added using a register identification model trained on the FinCORE corpus (Skantsi & Laippala, 2023). This corpus was originally compiled from a random sample of the Parsebank documents that have been manually annotated for text register.…”

Section: Registersmentioning

confidence: 99%

“…We used the FinBERT model (Virtanen et al, 2019) and the nine register categories in a multi-label setting, since a small proportion of the FinCORE documents are labeled with more than one main register category. The model fine-tuning was done following the best setting reported by Skantsi and Laippala (2023). Evaluation of the model on the FinCORE test data shows that the classifier achieves an F1-score of 82%.…”

Section: Registersmentioning

confidence: 99%

See 3 more Smart Citations

Finnish Internet Parsebank

Luotolahti

Kanerva

Luoma

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

We present a Finnish web corpus with multiple text sources and rich additional annotations. The corpus is based in large parts on a dedicated Internet crawl, supplementing data from the Common Crawl initiative and the Finnish Wikipedia. The size of the corpus is 6.2 billion tokens from 9.5 million source documents. The text is enriched with morphological analyses, word lemmas, dependency trees, named entities and text register (genre) identification. Paragraph-level scores of an n-gram language model, as well as paragraph duplication rate in each document are provided, allowing for further filtering of the dataset by the end user. Thanks to changes in the 2023 Finnish copyright legislation, the corpus is openly available for research purposes, and can also be accessed through the NoSketchEngine concordance tool and the dep search dependency tree query tool, all at https://turkunlp.org/finnish nlp.html.

show abstract

Section: Removal Of Machine-translated Contentmentioning

confidence: 99%

Section: Linguistic Researchmentioning

confidence: 99%

Section: Registersmentioning

confidence: 99%

Section: Registersmentioning

confidence: 99%

See 2 more Smart Citations

Finnish Internet Parsebank

Luotolahti

Kanerva

Luoma

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…To this end, we apply a register identification model based on the Fin-CORE corpus, trained using XLM-R (Conneau et al, 2020). The model and corpus were both presented by Skantsi and Laippala (2022). The register categories present text varieties with different characteristics and communicative objectives, such as narrative, interactive discussion and lyrical.…”

Section: Register Analysismentioning

confidence: 99%

FinGPT: Large Generative Models for a Small Language

Luukkonen,

Komulainen,

Luoma

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Large language models (LLMs) excel in many tasks in NLP and beyond, but most open models have very limited coverage of smaller languages and LLM work tends to focus on languages where nearly unlimited data is available for pretraining. In this work, we study the challenges of creating LLMs for Finnish, a language spoken by less than 0.1% of the world population. We compile an extensive dataset of Finnish combining web crawls, news, social media and eBooks. We pursue two approaches to pretrain models: 1) we train seven monolingual models from scratch (186M to 13B parameters) dubbed FinGPT, 2) we continue the pretraining of the multilingual BLOOM model on a mix of its original training data and Finnish, resulting in a 176 billion parameter model we call BLUUMI. For model evaluation, we introduce FIN-bench, a version of BIG-bench with Finnish tasks. We also assess other model qualities such as toxicity and bias. Our models and tools are openly available at https://turkunlp.org/gpt3-finnish.

show abstract

Automatic genre identification: a survey

Kuzman,

Ljubešić

2023

Lang Resources & Evaluation

View full text Add to dashboard Cite

Automatic genre identification (AGI) is a text classification task focused on genres, i.e., text categories defined by the author’s purpose, common function of the text, and the text’s conventional form. Obtaining genre information has been shown to be beneficial for a wide range of disciplines, including linguistics, corpus linguistics, computational linguistics, natural language processing, information retrieval and information security. Consequently, in the past 20 years, numerous researchers have collected genre datasets with the aim to develop an efficient genre classifier. However, their approaches to the definition of genre schemata, data collection and manual annotation vary substantially, resulting in significantly different datasets. As most AGI experiments are dataset-dependent, a sufficient understanding of the differences between the available genre datasets is of great importance for the researchers venturing into this area. In this paper, we present a detailed overview of different approaches to each of the steps of the AGI task, from the definition of the genre concept and the genre schema, to the dataset collection and annotation methods, and, finally, to machine learning strategies. Special focus is dedicated to the description of the most relevant genre schemata and datasets, and details on the availability of all of the datasets are provided. In addition, the paper presents the recent advances in machine learning approaches to automatic genre identification, and concludes with proposing the directions towards developing a stable multilingual genre classifier.

show abstract

Analyzing the unrestricted web: The finnish corpus of online registers

Cited by 5 publications

References 63 publications

Finnish Internet Parsebank

Finnish Internet Parsebank

FinGPT: Large Generative Models for a Small Language

Automatic genre identification: a survey

Contact Info

Product

Resources

About