OSU at SigMorphon 2022: Analogical Inflection With Rule Features

Elsner, Micha; Court, Sara

doi:10.18653/v1/2022.sigmorphon-1.22

Cited by 5 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, data augmentation (Anastasopoulos and Neubig, 2019;Silfverberg et al, 2017) can also improve the performance of models in low-resource languages. Seq2seq models, such as RNN+attention (Wiemerslage et al, 2018) or Transformer (Yang et al, 2022;Merzhevich et al, 2022;Elsner and Court, 2022), have become popular framework for morphological inflection in recent years. The lemma and tags are usually input together in this framework, and the model generates the inflected word.…”

Section: Related Workmentioning

confidence: 99%

Strategies to Improve Low-Resource Agglutinative Languages Morphological Inflection

Abudouwaili,

Ablez,

Abiderexiti

et al. 2023

Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL)

View full text Add to dashboard Cite

Morphological inflection is a crucial task in the field of morphology and is typically considered a sequence transduction task. In recent years, it has received substantial attention from researchers and made significant progress. Models have achieved impressive performance levels for both high-and low-resource languages. However, when the distribution of instances in the training dataset changes, or novel lemma or feature labels are predicted, the model's accuracy declines. In agglutinative languages, morphological inflection involves phonological phenomena while generating new words, which can alter the syllable patterns at the boundary between the lemma and the suffixes. This paper proposes four strategies for low-resource agglutinative languages to enhance the model's generalization ability. Firstly, a convolution module extracts syllable-like units from lemmas, allowing the model to learn syllable features. Secondly, the lemma and feature labels are represented separately in the input, and the position encoding of the feature labels is marked so that the model learns the order between suffixes and labels. Thirdly, the model recognizes the common substrings in lemmas through two special characters and copies them into words. Finally, combined with syllable features, we improve the data augmentation method. A series of experiments show that the proposed model in this paper is superior to other baseline models.

show abstract

Section: Related Workmentioning

confidence: 99%

Strategies to Improve Low-Resource Agglutinative Languages Morphological Inflection

Abudouwaili,

Ablez,

Abiderexiti

et al. 2023

Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL)

View full text Add to dashboard Cite

show abstract

“…Additionally, data augmentation (Anastasopoulos and Neubig, 2019; Silfverberg et al, 2017) can also improve the performance of models in low-resource languages. Seq2seq models, such as RNN+attention (Wiemerslage et al, 2018) or Transformer Merzhevich et al, 2022;Elsner and Court, 2022), have become popular framework for morphological inflection in recent years. The lemma and tags are usually input…”

Section: Acknowledgementsmentioning

confidence: 99%

Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL)

2023

View full text Add to dashboard Cite

Language models (LMs) have been argued to overlap substantially with human beings in grammaticality judgment tasks. But when humans systematically make errors in language processing, should we expect LMs to behave like cognitive models of language and mimic human behavior? We answer this question by investigating LMs' more subtle judgments associated with "language illusions" -sentences that are vague in meaning, implausible, or ungrammatical but receive unexpectedly high acceptability judgments by humans. We looked at three illusions: the comparative illusion (e.g. "More people have been to Russia than I have"), the depth-charge illusion (e.g. "No head injury is too trivial to be ignored"), and the negative polarity item (NPI) illusion (e.g. "The hunter who no villager believed to be trustworthy will ever shoot a bear"). We found that probabilities represented by LMs were more likely to align with human judgments of being "tricked" by the NPI illusion which examines a structural dependency, compared to the comparative and the depth-charge illusions which require sophisticated semantic understanding. No single LM or metric yielded results that are entirely consistent with human behavior. Ultimately, we show that LMs are limited both in their construal as cognitive models of human language processing and in their capacity to recognize nuanced but critical information in complicated language materials.

show abstract

“…The authors reported their best average accuracy of 55.5% when the number of hallucinated data is 1000. Elsner and Court (2022) augment each training instance with a representative exemplar lemma and its output form. Training instances are also augmented with rule-based features constructed by aligning source lemma with its inflected form and exemplar lemma with its inflected form.…”

Section: Related Workmentioning

confidence: 99%

Neural Arabic singular-to-plural conversion using a pretrained Character-BERT and a fused transformer

Radman,

Atros,

Duwairi

2023

Nat. Lang. Eng.

View full text Add to dashboard Cite

Morphological re-inflection generation is one of the most challenging tasks in the natural language processing (NLP) domain, especially with morphologically rich, low-resource languages like Arabic. In this research, we investigate the ability of transformer-based models in the singular-to-plural Arabic noun conversion task. We start with pretraining a Character-BERT model on a masked language modeling task using 1,134,950 Arabic words and then adopting the fusion technique to transfer the knowledge gained by the pretrained model to a full encoder–decoder transformer model, in one of the proposed settings. The second proposed setting directly fuses the output Character-BERT embeddings into the decoder. We then analyze and compare the performance of the two architectures and provide an interpretability section in which we track the features of attention with respect to the model. We perform the interpretation on both the macro and micro levels, providing some individual examples. Moreover, we provide a thorough error analysis showing the strengths and weaknesses of the proposed framework. To the best of our knowledge, this is the first effort in the Arabic NLP domain that adopts the development of an end-to-end fused-transformer deep learning model to address the problem of singular-to-plural conversion.

show abstract

OSU at SigMorphon 2022: Analogical Inflection With Rule Features

Cited by 5 publications

References 17 publications

Strategies to Improve Low-Resource Agglutinative Languages Morphological Inflection

Strategies to Improve Low-Resource Agglutinative Languages Morphological Inflection

Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL)

Neural Arabic singular-to-plural conversion using a pretrained Character-BERT and a fused transformer

Contact Info

Product

Resources

About