Sign constraints on feature weights improve a joint model of word segmentation and phonology

Abstract

This paper describes a joint model of word segmentation and phonological alternations, which takes unsegmented utterances as input and infers word segmentations and underlying phonological representations. The model is a Maximum Entropy or log-linear model, which can express a probabilistic version of Optimality Theory (OT; Prince and Smolensky (2004)), a standard phonological framework. The features in our model are inspired by OT's Markedness and Faithfulness constraints. Following the OT principle that such features indicate "violations", we require their weights to be non-positive. We apply our model to a modified version of the Buckeye corpus (Pitt et al., 2007) in which the only phonological alternations are deletions of word-final /d/ and /t/ segments. The model sets a new state-ofthe-art for this corpus for word segmentation, identification of underlying forms, and identification of /d/ and /t/ deletions. We also show that the OT-inspired sign constraints on feature weights are crucial for accurate identification of deleted /d/s; without them our model posits approximately 10 times more deleted underlying /d/s than appear in the manually annotated data.

A model of rapid phonotactic generalization

“…Like PAIM, but unlike humans, generalization to CONF-UNATT items diminished after multiple exposure sets (in particular for σ = 0.5). A straightforward implementation of MaxEnt is therefore incapable of simulating the human results; better results could potentially be achieved with a regularization method that encouraged sparsity (Goodman, 2004;Johnson et al, 2015).…”

Segmentation and UR Acquisition with UR Constraints

“…In other words, every underlying form must surface faithfully at least once in order to be considered a possible UR. This assumption is shared by other models of UR acquisition, such as Albright (2002), and of segmentation and UR acquisition (Johnson et al, 2015).…”

“…The fact that segmentation is a prerequisite to build the lexicon only precludes lexical information from being used in segmentation if the two processes take place in serial, with learners developing the ability to segment speech before storing any lexical information. Previous models of segmentation either ignore the acquisition of the lexicon (Saffran et al, 1996a;Saffran et al, 1996b;Perruchet and Vinter, 1998) or do not fully utilize the richness of lexical knowledge (Johnson et al, 2015;Goldwater et al, 2009). This paper presents a model of segmentation in which the lexicon, represented by phonological underlying forms which correspond to meanings, is being acquired in parallel with segmentation, and the two processes are mutually informing.…”

Coding Textual Inputs Boosts the Accuracy of Neural Networks

“…We emphasize that we do not use any additional information besides our algorithms. Hayes (1996); Johnson et al (2015) applied explicit phonological rules or constraints to tasks such as word segmentation. In neural networks, we can implicitly learn from phonetic data and leave the networks to discover hidden phonetic features through end-to-end training opt specific NLP tasks, instead of applying hand-coded constraints.…”