Sequencing and Optimization Within an Embodied Task Dynamic Model

Abstract

A model of gestural sequencing in speech is proposed that aspires to producing biologically plausible fluent and efficient movement in generating an utterance. We have previously proposed a modification of the well-known task dynamic implementation of articulatory phonology such that any given articulatory movement can be associated with a quantification of effort (Simko & Cummins, 2010). To this we add a quantitative cost that decreases as speech gestures become more precise, and hence intelligible, and a third cost component that places a premium on the duration of an utterance. Together, these three cost elements allow us to derive algorithmically optimal sequences of gestures and dynamical parameters for generating articulator movement. We show that the optimized movement displays many timing characteristics that are representative of real speech movement, capturing subtle details of relative timing between gestures. Optimal movement sequences also display invariances in timing that suggest syllable-level coordination for CV sequences. We explore the behavior of the model as prosodic context is manipulated in two dimensions: clarity of articulation and speech rate. Smooth, fluid, and efficient movements result.

Timing in talking: what is it used for, and how is it controlled?

“…Many models of motor control in the optimal control theory framework are like DIVA in that they assume that we continuously monitor the states of our effectors (e.g. their position and velocity) in relation to the task goals, continuously updating our motor commands on the basis of state information to accomplish goals in a near-optimal way (but see [70,71] for an exception). In these models, movements are generated via a control policy that determines the optimal movement from any current state given the task goals and costs of movement.…”

“…Simko & Cummins' embodied task dynamics model [70,71] is an interesting case: an example of a theory of speech motor control in which time is used only as a cost (where surface utterance duration is penalized), but not a parameter of movement. In avoiding the use of time as a parameter of movement, this model is similar to the articulatory phonology/task dynamics (AP/TD) approach, discussed in more detail below.…”

“…Simko & Cummins' embodied task dynamic model [70,71] is similar to AP/TD in that it uses mass-spring oscillator systems for gestures. In this model, some aspects of surface timing are emergent, i.e.…”

“…This evidence therefore supports models like DIVA/VITE in which duration is a planned parameter of movement. Although duration is not a parameter of movement in Simko & Cummins' [70,71] model, this model could probably be modified to account for these data, because Simko & Cummins use an extrinsic timekeeper to specify an utterance duration cost. However, their model might need to be amended to measure, represent and specify durations of constituents smaller than the utterance.…”

Context-dependent articulation of consonant gemination in Estonian

“…between the relevant position measure (position of the TB sensor, value of the derived 'lip aperture' measure) at the onset and offset of the gesture. In addition, the lip-tongue coordination measure was computed as the onset time of the vowel transition movement minus the onset time of the bilabial closing movement (see Löfqvist & Gracco, 1999;Šimko & Cummins, 2010Šimko & Cummins, , 2011Šimko et al, 2014).…”