Decoding behavior from global cerebrovascular activity using neural networks

Abstract

Functional Ultrasound (fUS) provides spatial and temporal frames of the vascular activity in the brain with high resolution and sensitivity in behaving animals. The large amount of resulting data is underused at present due to the lack of appropriate tools to visualize and interpret such signals. Here we show that neural networks can be trained to leverage the richness of information available in fUS datasets to reliably determine behavior, even from a single fUS 2D image after appropriate training. We illustrate the potential of this method with two examples: determining if a rat is moving or static and decoding the animal’s sleep/wake state in a neutral environment. We further demonstrate that our method can be transferred to new recordings, possibly in other animals, without additional training, thereby paving the way for real-time decoding of brain activity based on fUS data. Finally, the learned weights of the network in the latent space were analyzed to extract the relative importance of input data to classify behavior, making this a powerful tool for neuroscientific research.

Decoding motor plans using a closed-loop ultrasonic brain–machine interface

“…Aside from the amount of data required to train ANNs, recent work has highlighted additional challenges in training deep learning models for closed-loop motor BMI control 37 , especially avoiding overfitting of the model to the temporal structure in previously recorded data. Although training and inference using ANNs were beyond the scope of the current experiments, this could become an important area of investigation for future, more sophisticated fUS-BMIs 38 .…”

The Emergence of Functional Ultrasound for Noninvasive Brain–Computer Interface

“…Berthon et al. [ 94 ] used artificial neural network to decode the global blood-flow information obtained from fUS to decode the behavior of the brain, and their network was able to accurately predict the movement or resting of rats, demonstrating the potential of artificial neural network in BCI based on fUS.…”

“…The differences in brain anatomy between the 2 subjects, or different relative positions of the probes during the 2 acquisitions, can result in pixels not being matched to the corresponding anatomical regions, thus affecting decoding outcome by the decoder. A perfect match between the 2 sets of data is required to achieve the correct decoding and encoding of brain information [ 94 ]. Thanks to the 3D ultrasound, whole-brain imaging can be achieved to reduce the impact of differences in brain slice acquisition, but the huge data volume of 3D ultrasound may limit the BCI's online application.…”