Canonical and Surface Morphological Segmentation for Nguni Languages

Algorithm for Assisting Grammarians when Extracting Phonological Conditioning Rules for Nguni languages

“…When widening the search for related literature further, we see that there exists a verb parser and generator (Pretorius et al (2017)), morphological analysers (Pretorius & Bosch (2009), ), morphological segmenters (e.g., Mzamo et al (2019), Moeng et al (2021)), language models (e.g., Myoya et al (2023)), and a Grammatical Framework (GF) grammar…”

“…Specifically, the morphological analysers and canonical segmenters take a word as input and produce canonical morphemes, hence they need to capture phonological conditioning rules, even if that is in an implicit manner, to be able to reverse them. For instance, Moeng et al (2021)'s models can only generate the canonical morphemes nga-i-zin-konzo when given ngezinkonzo 'by the services' if they model the reversal of the phonological conditioning rule a + i → e. These resources also cannot uncover new rules and additional limitations to this type of work are as follows:…”

“…Since the languages are underresourced and grammatically complex, prevalent neural text generation architectures are likely to struggle to capture the linguistic phenomena when forming individual words. This is implied by the performance of neural models created to undo phonological conditioning -canonical segmentation in Nguni languages (Moeng et al (2021)). Moeng et al (2021)'s best performing canonical segmentation models have F1 scores around 0.7 [3].…”

“…This is implied by the performance of neural models created to undo phonological conditioning -canonical segmentation in Nguni languages (Moeng et al (2021)). Moeng et al (2021)'s best performing canonical segmentation models have F1 scores around 0.7 [3]. However, analysis of the code [4] shows that the true performance is likely to be much lower since the quantification of the performance does not take the order and completeness of the morphemes into account.…”

A Masked Segmental Language Model for Unsupervised Natural Language Segmentation

“…Nevertheless, BPE does not necessarily help in situations where knowing a sensical segmentation of linguistic-like units is important, such as attempting to model the ways in which children acquire language (Goldwater et al, 2009), segmenting free-flowing speech (Kamper et al, 2016;Rasanen and Blandon, 2020), creating linguistic tools for morphologically complex languages (Moeng et al, 2021), or studying the structure of an endangered language with few or no current speakers (Dunbar et al, 2020).…”