Color in motion: Generating 3-dimensional multispectral models to study dynamic visual signals in animals

Abstract

Analyzing color and pattern in the context of motion is a central and ongoing challenge in the quantification of animal coloration. Many animal signals are spatially and temporally variable, but traditional methods fail to capture this dynamism because they use stationary animals in fixed positions. To investigate dynamic visual displays and to understand the evolutionary forces that shape dynamic colorful signals, we require cross-disciplinary methods that combine measurements of color, pattern, 3-dimensional (3D) shape, and motion. Here, we outline a workflow for producing digital 3D models with objective color information from museum specimens with diffuse colors. The workflow combines multispectral imaging with photogrammetry to produce digital 3D models that contain calibrated ultraviolet (UV) and human-visible (VIS) color information and incorporate pattern and 3D shape. These “3D multispectral models” can subsequently be animated to incorporate both signaler and receiver movement and analyzed in silico using a variety of receiver-specific visual models. This approach—which can be flexibly integrated with other tools and methods—represents a key first step toward analyzing visual signals in motion. We describe several timely applications of this workflow and next steps for multispectral 3D photogrammetry and animation techniques.

Recording animal-view videos of the natural world using a novel camera system and software package

“…First, as the method does not require switching parts (e.g., filters or lenses) between taking UV and visible images, which is common in other applications of multispectral photography, it allows for more rapid image collection ( S2 – S4 Videos). Therefore, this approach can speed up data acquisition in the pipelines of color estimation and photogrammetry projects [ 37 , 40 ]. Second, signals and displays can be studied in their natural contexts.…”

Recording animal-view videos of the natural world

“…filters or lenses) common in other applications of multispectral photography, it allows for more rapid data acquisition (Figure 3d). This can speed up the pipelines of color estimation and photogrammetry projects (Miller et al 2022; Medina et al 2020). Second, signals and displays can be studied in their natural context.…”

“…Yet temporal changes can be central to a biological signal (Rosenthal 2007; Hutton et al 2015). A recently proposed approach for studying dynamic color signals combines multispectral imaging with digital 3D modeling (Miller et al 2022). The next step will be to animate the resulting 3D multispectral models and further analyze them in silico , simulating a variety of receiver-specific visual models or viewing conditions (Kim et al 2012, Medina et al 2020, Miller et al 2022).…”

“…A recently proposed approach for studying dynamic color signals combines multispectral imaging with digital 3D modeling (Miller et al 2022). The next step will be to animate the resulting 3D multispectral models and further analyze them in silico , simulating a variety of receiver-specific visual models or viewing conditions (Kim et al 2012, Medina et al 2020, Miller et al 2022). Such models will likely play a crucial role in our understanding of the ways animal posture and the receivers’ viewpoints alter visual signals.…”

Habitat geometry rather than visual acuity limits the visibility of a ground‐nesting bird's clutch to terrestrial predators

“…Partial and self-occlusion will reduce the visible area of the clutch and mask recognizable features such as the clutch's shadow and edge (Lovell et al, 2013;Webster, 2015). Meanwhile, changes in back- (Miller et al, 2022;Vasas et al, 2022). However, these methods are slower and more computationally expensive than our 3D phone scans.…”