Development and application of a robotic zebra finch (RoboFinch) to study multimodal cues in vocal communication

Abstract

Understanding animal behaviour through psychophysical experimentation is often limited by insufficiently realistic stimulus representation. Important physical dimensions of signals and cues, especially those that are outside the spectrum of human perception, can be difficult to standardize and control separately with currently available recording and displaying techniques (e.g. video displays). Accurate stimulus control is in particular important when studying multimodal signals, as spatial and temporal alignment between stimuli is often crucial. Especially for audiovisual presentations, some of these limitations can be circumvented by the employment of animal robots that are superior to video presentations in all situations requiring realistic 3D presentations to animals. Here we report the development of a robotic zebra finch, called RoboFinch, and how it can be used to study vocal learning in a songbird, the zebra finch.

Cross-species parallels in babbling: animals and algorithms

“…In the case of systems that ‘learn’, recent times have seen huge developments in the fields of artificial intelligence and machine learning, primarily arising from advances in multi-layered artificial neural networks, an approach known as ‘deep learning’ [ 73 ]. It is, therefore, no surprise that a few researchers have started to apply these techniques to problems in bioacoustics [ 74 ], particularly for automatic call detection and classification [ 75 , 76 ]. However, as yet, there are few studies that apply such algorithms to vocal learning itself, and of those, all have been concerned with modelling the acquisition of vocal abilities by humans, and none, to our knowledge, has addressed vocal learning in other animals or in a general cross-species approach.…”

Driving singing behaviour in songbirds using multi-modal, multi-agent virtual reality

“…What passes for an expressive robot is species and hypothesis dependent, but many animals will readily accept a robot as part of their social network (Michelsen et al, 1992;Halloy et al, 2007;de Margerie et al, 2011;Romano et al, 2017). Building and controlling a small expressive robot might be possible in some cases (Simon et al, 2019) but is often not a viable solution for small model animals due to the mechanical and computational complexity involved in fully mimicking natural behaviours.…”

“…To situate a zebra finch in virtual reality requires at least sound and vision but is likely also influenced by gaze (Davidson and Clayton, 2016) and orientation relative to other agents (Ljubičić et al, 2016). Previous work has shown that zebra finches interact vocally with an immobile physical decoy providing audio from a built-in speaker (Benichov et al, 2016;Benichov and Vallentin, 2020) and are physically attracted to more life-like actuated zebra finch robots (Simon et al, 2019).…”

Driving singing behaviour in songbirds using a multi-modal, multi-agent virtual environment

“…What passes for an expressive robot is species and hypothesis dependent, but many animals will readily accept a robot as part of their social network [11][12][13][14] . Building and controlling a small expressive robot might be possible in some cases 15 but is often not a viable solution for small model animals due to the mechanical and computational complexity involved in fully mimicking natural behaviours.…”

“…To situate a zebra finch in a virtual environment requires at least sound and vision but is likely also influenced by gaze 29 and orientation relative to other agents 30 . Previous work has shown that zebra finches interact vocally with an immobile physical decoy providing audio from a built-in speaker 10,31 and are physically attracted to more life-like actuated zebra finch robots 15 . Furthermore, adult finches can recognize and discriminate between conspecifics from live video feeds 32,33 and sing song to still images 34 or live video feeds of females 34,35 .…”