Efficient Photoacoustic Image Synthesis with Deep Learning

Abstract

Photoacoustic imaging potentially allows for the real-time visualization of functional human tissue parameters such as oxygenation but is subject to a challenging underlying quantification problem. While in silico studies have revealed the great potential of deep learning (DL) methodology in solving this problem, the inherent lack of an efficient gold standard method for model training and validation remains a grand challenge. This work investigates whether DL can be leveraged to accurately and efficiently simulate photon propagation in biological tissue, enabling photoacoustic image synthesis. Our approach is based on estimating the initial pressure distribution of the photoacoustic waves from the underlying optical properties using a back-propagatable neural network trained on synthetic data. In proof-of-concept studies, we validated the performance of two complementary neural network architectures, namely a conventional U-Net-like model and a Fourier Neural Operator (FNO) network. Our in silico validation on multispectral human forearm images shows that DL methods can speed up image generation by a factor of 100 when compared to Monte Carlo simulations with 5×108 photons. While the FNO is slightly more accurate than the U-Net, when compared to Monte Carlo simulations performed with a reduced number of photons (5×106), both neural network architectures achieve equivalent accuracy. In contrast to Monte Carlo simulations, the proposed DL models can be used as inherently differentiable surrogate models in the photoacoustic image synthesis pipeline, allowing for back-propagation of the synthesis error and gradient-based optimization over the entire pipeline. Due to their efficiency, they have the potential to enable large-scale training data generation that can expedite the clinical application of photoacoustic imaging.

Distribution-informed and wavelength-flexible data-driven photoacoustic oximetry

“… predicts a plausible training dataset for all three in vivo applications tested in this study, where the predicted value range was 0.4 to 0.7 on the mouse data, 0.5 to 0.8 on the forearm data, and 0.3 to 0.7 on the data. With the development of fast and auto-differentiable simulation pipelines, 62 it should be possible to optimize the simulation parameters for accurate estimates by iteratively minimizing . When using differentiable implementations of distribution distance measures, it might even be possible to integrate this optimization into an unsupervised training routine.…”

Deep learning acceleration of iterative model-based light fluence correction for photoacoustic tomography

“…The reason for choosing U-Net as a comparison is that it is a common architecture for image-to-image learning tasks, especially in medical imaging, and a fully convolutional neural network compared to the FNO. Moreover, U-Net is already used for light transport in tissue [30] , [31] . In addition, we also train FNO as an end-to-end processing method, which is denoted as FNO-E2E.…”

“…Their performance is still inferior to traditional iterative correction approaches. In addition, deep learning also has been used to learn the light transport models and enhance the synthesis of photoacoustic image [30] , [31] , which is helpful for generating efficient training data for quantitative imaging.…”

Deep Learning-Based Super-Resolution Reconstruction and Segmentation of Photoacoustic Images

“…However, despite the significant progress made by deep-learning methods in the field of PAI processing, there are still some challenges and limitations. Because PAIs have complex structures and noise interference, traditional convolutional neural networks often perform poorly in processing PAIs, in which it is difficult to accurately reconstruct and segment the fine structures, and may suffer from overfitting or underfitting problems, resulting in insufficient model generalization ability [29]. Second, the application breadth and performance of deep-learning methods are limited because deep-learning models need a significant amount of labelled data for training, and the labelled data of PAI are hard to collect.…”