Using LSTMs to Assess the Obligatoriness of Phonological Distinctive Features for Phonotactic Learning

Abstract

To ascertain the importance of phonetic information in the form of phonological distinctive features for the purpose of segmentlevel phonotactic acquisition, we compare the performance of two recurrent neural network models of phonotactic learning: one that has access to distinctive features at the start of the learning process, and one that does not. Though the predictions of both models are significantly correlated with human judgments of non-words, the feature-naive model significantly outperforms the feature-aware one in terms of probability assigned to a held-out test set of English words, suggesting that distinctive features are not obligatory for learning phonotactic patterns at the segment level.

An algorithm for learning phonological classes from distributional similarity

“…The goal of this paper is to expand on the successes of this ongoing collective research programme. The algorithm described below shares many aspects with past work, such as vector embedding (Powers 1997, Calderone 2009, Goldsmith & Xanthos 2009, Nazarov 2014, 2016, Silfverberg et al 2018, Mirea & Bicknell 2019, normalisation (Powers 95 Learning phonological classes from distributional similarity 1997, Silfverberg et al 2018), matrix decomposition (Powers 1997, Calderone 2009, Goldsmith & Xanthos 2009, Silfverberg et al 2018 and clustering algorithms (Powers 1997, Nazarov 2014, 2016, Mirea & Bicknell 2019. The innovations that will be presented below are largely in the combination and extension of these techniques, but the clustering methodology presented is relatively novel.…”

“…Their use of maximum entropy Hidden Markov Models also involves a kind of one-dimensional clustering on emission probability ratios, setting a threshold of 0 as the boundary between clusters. Powers (1997) and Mirea & Bicknell (2019) both use hierarchical clustering to extract classes from embeddings. Hierarchical clustering is simple, but not well suited to phonological class discovery: it cannot find multiple partitions of the same set of sounds, and requires the number of classes to be decided by an analyst.…”

“…I include Samoan because it has a relatively small segmental inventory and fairly restrictive phonotactics, providing a simple test case. English, French and Finnish are included for continuity with previous studies (Calderone 2009, Goldsmith & Xanthos 2009, Silfverberg et al 2018, Mirea & Bicknell 2019. In addition to simply exploring which classes are distributionally salient in each language, my 14 A reviewer wonders, following Archangeli et al (2011), whether certain types of noise are more disruptive to this algorithm than others, and whether these correspond to what we see in natural language.…”

“…Their non-neural comparison model employs vector embedding, singular value decomposition, and normalisation using positive pointwise mutual information, all of which are used in the algorithm presented in this paper, but it does not take phoneme ordering into account. Similarly, Mirea & Bicknell (2019) generate phoneme embeddings using a long-term short-term memory neural network, and perform hierarchical clustering on these embeddings. This clustering does not cleanly separate consonants and vowels, though some suggestive groupings are present.…”

“…It is likely that considering additional aspects of context would improve performance on the real languages, although simply increasing the size of the contexts considered in an n-gram model will lead to issues of data paucity. Using sequential neural networks to generate phoneme embeddings is a particularly promising possibility, since they can produce vector representations of sounds without being explicitly told which features of the context to attend to (Silfverberg et al 2018, Mirea & Bicknell 2019, Mayer & Nelson 2020. Alternatively, integrating a mechanism for tier projection (e.g.…”