Learned context dependent categorical perception in a songbird

Sainburg, Tim; Thielk, Marvin; Gentner, Timothy Q.

doi:10.32470/ccn.2018.1147-0

Cited by 4 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The utility of non-linear dimensionality reduction techniques are just now coming to fruition in the study of animal communication, for example using t-distributed stochastic neighborhood embedding (t-SNE; [32]) to describe the development of zebra finch song [34], using Uniform Manifold Approximation and Projection (UMAP; [31]) to describe and infer categories in birdsong [3,35], or using deep neural networks to synthesize naturalistic acoustic stimuli [36,37]. Developments in non-linear representation learning have helped fuel the most recent advancements in machine learning, untangling statistical relationships in ways that provide more explanatory power over data than traditional linear techniques [13,14].…”

Section: Latent Models Of Acoustic Communicationmentioning

confidence: 99%

“…For example, the latent spaces of some neural networks linearize the presence of a beard in an image of a face without being trained on beards in any explicit way [15,44]. Complex features of vocalizations are similarly captured in intuitive ways in latent projections [3,[35][36][37]. Depending on the organization of the dataset projected into a latent space, these features can extend over biologically or psychologically relevant scales.…”

Section: Discrete Latent Projections Of Animal Vocalizationsmentioning

confidence: 99%

“…Generative techniques enable the synthesis of complex, high-dimensional data such as animal communication signals by sampling from lowdimensional latent spaces. Preliminary work has already been done in this area, for example, generating syllables of birdsong as stimuli for psychophysical and neurophysiological probes [36,37] using deep neural networks. HMMs have also been used to synthesize vocalizations [93].…”

Section: Future Workmentioning

confidence: 99%

See 2 more Smart Citations

Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires

2020

Self Cite

View full text Add to dashboard Cite

Animals produce vocalizations that range in complexity from a single repeated call to hundreds of unique vocal elements patterned in sequences unfolding over hours. Characterizing complex vocalizations can require considerable effort and a deep intuition about each species’ vocal behavior. Even with a great deal of experience, human characterizations of animal communication can be affected by human perceptual biases. We present a set of computational methods for projecting animal vocalizations into low dimensional latent representational spaces that are directly learned from the spectrograms of vocal signals. We apply these methods to diverse datasets from over 20 species, including humans, bats, songbirds, mice, cetaceans, and nonhuman primates. Latent projections uncover complex features of data in visually intuitive and quantifiable ways, enabling high-powered comparative analyses of vocal acoustics. We introduce methods for analyzing vocalizations as both discrete sequences and as continuous latent variables. Each method can be used to disentangle complex spectro-temporal structure and observe long-timescale organization in communication.

show abstract

Section: Latent Models Of Acoustic Communicationmentioning

confidence: 99%

Section: Discrete Latent Projections Of Animal Vocalizationsmentioning

confidence: 99%

Section: Future Workmentioning

confidence: 99%

See 1 more Smart Citation

Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…This data-driven approach is closely related to previous studies that have applied autoencoding to birdsong for purposes of generating spectrograms and interpolating syllables for use in playback experiments [38, 44]. Additionally, dimensionality reduction algorithms such as the UMAP [29] and t-SNE [27] algorithms we use here to visualize latent spaces have previously been applied to raw spectrograms of birdsong syllables to aid in syllable clustering [37] and to visualize juvenile song learning [25].…”

Section: Discussionmentioning

confidence: 95%

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Goffinet

Brudner

Mooney

2019

Preprint

View full text Add to dashboard Cite

Vocalization is an essential medium for social and sexual signaling in most birds and mammals. 1 Consequently, the analysis of vocal behavior is of great interest to fields such as neuroscience and 2 linguistics. A standard approach to analyzing vocalization involves segmenting the sound stream 3 into discrete vocal elements, calculating a number of handpicked acoustic features, and then using 4 the feature values for subsequent quantitative analysis. While this approach has proven powerful, 5 it suffers from several crucial limitations: First, handpicked acoustic features may miss important 6 dimensions of variability that are important for communicative function. Second, many analyses 7 assume vocalizations fall into discrete vocal categories, often without rigorous justification. Third, a 8 syllable-level analysis requires a consistent definition of syllable boundaries, which is often difficult 9 to maintain in practice and limits the sorts of structure one can find in the data. To address these 10 shortcomings, we apply a data-driven approach based on the variational autoencoder (VAE), an 11 unsupervised learning method, to the task of characterizing vocalizations in two model species: 12 the laboratory mouse (Mus musculus) and the zebra finch (Taeniopygia guttata). We find that the 13 VAE converges on a parsimonious representation of vocal behavior that outperforms handpicked 14 acoustic features on a variety of common analysis tasks, including representing acoustic similarity and 15 recovering a known effect of social context on birdsong. Additionally, we use our learned acoustic 16 features to argue against the widespread view that mouse ultrasonic vocalizations form discrete 17 syllable categories. Lastly, we present a novel "shotgun VAE" that can quantify moment-by-moment 18 variability in vocalizations. In all, we show that data-derived acoustic features confirm and extend 19 existing approaches while offering distinct advantages in several critical applications. 20 1 Introduction 21 Vocalization is an essential medium for social and sexual signaling in most birds and mammals, and also serves as a 22 natural substrate for language and music in humans. Consequently, the analysis of vocal behavior is of great interest to 23 ethologists, psychologists, linguists, and neuroscientists. A major goal of these various lines of enquiry is to develop 24 methods for quantitative analysis of vocal behavior, efforts that have resulted in several powerful methods that enable 25 the automatic or semi-automatic analysis of vocalizations. Key to this approach has been the existence of software 26 packages that calculate acoustic features for each syllable within a vocalization [4, 39, 40, 7, 6]. For example, Sound 27 Analysis Pro, focused on birdsong, calculates 14 features for each syllable, including duration, spectral entropy, and 28 goodness of pitch, and uses these as a basis for subsequent clustering and analysis [39]. More recently, MUPET and 29 DeepSqueak have applied a similar approach to mouse vocalizati...

show abstract

“…These neural-network-based algorithms can be used to sample directly from the learned representational spaces described in Section 3. A simple example is autoencoder-based synthesis ( Figure 6 ) (Sainburg et al, 2018a ; Zuidema et al, 2020 ). Autoencoders can be trained on spectral representations of vocal data, and systematically sampled in the learned latent space to produce new vocalizations.…”

Section: Synthesizing Vocalizationsmentioning

confidence: 99%

Toward a Computational Neuroethology of Vocal Communication: From Bioacoustics to Neurophysiology, Emerging Tools and Future Directions

Sainburg

Gentner

2021

Front. Behav. Neurosci.

Self Cite

View full text Add to dashboard Cite

Recently developed methods in computational neuroethology have enabled increasingly detailed and comprehensive quantification of animal movements and behavioral kinematics. Vocal communication behavior is well poised for application of similar large-scale quantification methods in the service of physiological and ethological studies. This review describes emerging techniques that can be applied to acoustic and vocal communication signals with the goal of enabling study beyond a small number of model species. We review a range of modern computational methods for bioacoustics, signal processing, and brain-behavior mapping. Along with a discussion of recent advances and techniques, we include challenges and broader goals in establishing a framework for the computational neuroethology of vocal communication.

show abstract

Learned context dependent categorical perception in a songbird

Cited by 4 publications

References 0 publications

Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires

Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires

Low-dimensional learned feature spaces quantify individual and group differences in vocal repertoires

Toward a Computational Neuroethology of Vocal Communication: From Bioacoustics to Neurophysiology, Emerging Tools and Future Directions

Contact Info

Product

Resources

About