Reconstructing Undersampled Photoacoustic Microscopy Images Using Deep Learning

Abstract

Oneprimary technical challenge in photoacoustic microscopy (PAM) is the necessary compromise between spatial resolution and imaging speed. In this study, we propose a novel application of deep learning principles to reconstruct undersampled PAM images and transcend the trade-off between spatial resolution and imaging speed. We compared various convolutional neural network (CNN) architectures, and selected a fully dense U-net (FD U-net) model that produced the best results. To mimic various undersampling conditions in practice, we artificially downsampled fullysampled PAM images of mouse brain vasculature at different ratios. This allowed us to not only definitively establish the ground truth, but also train and test our deep learning model at various imaging conditions. Our results and numerical analysis have collectively demonstrated the robust performance of our model to reconstruct PAM images with as few as 2% of the original pixels, which may effectively shorten the imaging time without substantially sacrificing the image quality.

Deep image prior for undersampling high-speed photoacoustic microscopy

“…Consistent with the reported FD U-Net in [ 29 ], the input of DIP has 128 128 pixels. DIP FD U-Net [ 29 ] Training time 2.60 min 8 h Inference time -- 46.0 ms …”

“…These metrics represent both global and local information of the reconstructed fully-sampled images [ 25 ]. SSIM and PSNR were also computed for bilinear, bicubic, and lanczos (8 8 kernel) interpolation as well as our recently published pre-trained FD U-Net [ 29 ]. In FD U-Net, each block at a certain spatial level has channel-wise concatenation of all convolutional layers as the output, which forms “collective knowledge” that is passed to the next block [ 29 , 46 ].…”

“…To accommodate large images ( i.e. , > 300 300 pixels), we used a model patchwork algorithm to avoid border artifacts [ 29 ]. Given enough GPU memory, it is also possible to process the whole image at once without patching.…”

“…Over the past few years, researchers have primarily focused on applying DL in photoacoustic computed tomography (PACT) for artifact removal, target identification, and sparse sampling [ [18] , [19] , [20] , [21] , [22] , [23] , [24] , [25] , [26] ]. Recently, DL has been used to reduce the laser pulse energy [ 27 ] and improve the undersampling in PAM [ 28 , 29 ]. For the undersampling issue, previous applications have demonstrated promising performance on in vivo PAM data, with a structural similarity index (SSIM) of up to 0.92 [ 28 , 29 ].…”

“…Recently, DL has been used to reduce the laser pulse energy [ 27 ] and improve the undersampling in PAM [ 28 , 29 ]. For the undersampling issue, previous applications have demonstrated promising performance on in vivo PAM data, with a structural similarity index (SSIM) of up to 0.92 [ 28 , 29 ]. However, the previous DL methods typically depend on training convolutional neural networks (CNN) with a large training dataset as the ground truth.…”