Songs of the Bengalese finch consist of variable sequences of syllables. The sequences follow probabilistic rules, and can be statistically described by partially observable Markov models (POMMs), which consist of states and probabilistic transitions between them. Each state is associated with a syllable, and one syllable can be associated with multiple states. This multiplicity of syllable to states association distinguishes a POMM from a simple Markov model, in which one syllable is associated with one state. The multiplicity indicates that syllable transitions are context-dependent. Here we present a novel method of inferring a POMM with minimal number of states from a finite number of observed sequences. We apply the method to infer POMMs for songs of six adult male Bengalese finches before and shortly after deafening. Before deafening, the models all require multiple states, but with varying degrees of state multiplicity for individual birds. Deafening reduces the state multiplicity for all birds. For three birds, the models become Markovian, while for the other three, the multiplicity persists for some syllables. These observations indicate that auditory feedback contributes to, but is not the only source of, the context dependencies of syllable transitions in Bengalese finch song.