Animal-in-the-loop System with Multimodal Virtual Reality to Elicit Natural Olfactory Localization Behavior

Abstract

Herein, we proposed an "animal-in-the-loop" (AIL) system by introducing a robot and a virtual reality (VR) technology in a conventional insect behavior experiment. The setup provided sensory inputs to an insect, which mimic its natural environment and simultaneously measured the behavioral output. The proposed AIL system consisted of a multimodal VR device and a ground-running robot, both of which were connected wirelessly. The insect behavior was measured using a multimodal VR device, and the behavioral changes were transmitted to the robot as control inputs. Specifically, the multimodal VR device was equipped with three types of sensory stimulators, odor, wind, and vision, and each stimulator was controlled by the value of the corresponding sensor on the robot. The surrounding environment was observed using multiple sensors mounted on the robot, and the information was transmitted to the VR device to provide sensory stimuli to the insect. This system allowed the insect on the VR device to remotely control the robot and perform localization virtually. The localization trajectories of the proposed AIL system were similar to those of the free-walking experiment, and the tendency of the change in the heading angle during localization was also similar. Therefore, we found that using the AIL system enabled us to measure behavioral changes upon providing sensory stimuli to insects. These VR stimuli were similar to those encountered by the insect in free-walking experiments.

Usage of a Sensory-Motor Intervention System for Understanding the Adaptive Behavior of Insects

“…The experiments introduced here were conducted in the laboratory or a virtual environment, and we have not been able to measure the adaptive behavior of the insect in more natural conditions, such as in the outdoor environment, where odor distribution is more natural. To address this issue, one could consider using our proposed "Animal-in-the-Loop" system [76,77]. We have been unable to explore the level of behavioral modulation of the nervous system in insects in response to sensory and motor interventions because both sensory-motor intervention systems introduced here have been analyzed only at the behavioral level.…”

Robust Moth-Inspired Algorithm for Odor Source Localization Using Multimodal Information

“…Because the odor sensors have a low temporal resolution problem, we addressed this issue by using an inverse model-estimation method based on the ARX model [ 22 ]. The directionality of the wind sensors was increased by installing a partition between each wind sensor [ 18 , 23 ], allowing them to discriminate between four different directions.…”