Experimental investigation of neural network estimator and transfer learning techniques for K‐edge spectral CT imaging

Abstract

Purpose Spectral computed tomography (CT) material decomposition algorithms require accurate physics‐based models or empirically derived models. This study investigates a machine learning algorithm and transfer learning techniques for Spectral CT imaging of K‐edge contrast agents using simulated and experimental measurements. Methods A feed forward multilayer perceptron was implemented and trained on data acquired using a step wedge phantom containing acrylic, aluminum, and gadolinium materials. The neural network estimator was evaluated by scanning a rod phantom with varying dilutions of gadolinium oxide nanoparticles and by scanning a rat leg specimen with injected nanoparticles on a bench‐top photon‐counting computed tomography system. The algorithm decomposed each spectral projection measurement into path lengths of acrylic and aluminum and mass lengths of gadolinium. Each basis material sinogram was reconstructed into basis material images using filtered backprojection. Machine learning techniques of data standardization, transfer learning from aggregated pixel data, and transfer learning from simulations were investigated to improve image quality. The algorithm was compared to a previously published empirical method based on a linear approximation and calibration error look‐up tables. Results The combined transfer learning techniques did not improve quantification in the rod phantom and provided only a small qualitative improvement in ring artifacts. Transfer learning from aggregated pixel data and from simulations improved the qualitative image quality of the rat specimen, for which the calibration data were limited. Transfer learning from aggregated pixel data and simulations estimated 3.26, 6.26, and 12.45 mg/mL Gd concentrations compared to true 2.72, 5.44, and 10.88 mg/mL concentrations in the rod phantom. Additionally, the neural networks were able to separate the soft tissue, bone, and gadolinium nanoparticles of the ex vivo rat leg specimen into the different basis images. Conclusions The results demonstrate that empirical K‐edge imaging from calibration measurements using machine learning and transfer learning is possible without explicit models of material attenuations, incident spectra, or the detector response.

Material Decomposition in Spectral CT Using Deep Learning: A Sim2Real Transfer Approach

“…Most of them perform material decomposition in the image domain [9], [39]. Recently a multilayer perceptron was proposed for solving the material decomposition problem in the projection domain [42]. In this work, we proposed a CNN approach to assess the potential of deep learning not only for solving the material decomposition problem but for implicit regularization.…”

“…In [9], the authors proposed a VGG-16 network and tested it on Shepp-Logan synthetic data and experimental cylindrical phantoms. The previously proposed neural networks approaches in the projection domain have been based on multilayer perceptrons (using fully connected layers) for decomposing materials in a pixel-by-pixel basis [40], [41], [42]. In [40], the authors used a neural network with two hidden layers followed by a denoising method to mitigate noise.…”

“…Different approaches for TL can be found in [45], [46]; most common is to pre-train a model on a large-annotated data set and then to fine tune the model on a more specific and smaller data set [47], [48]. Recent works proposed related TL approaches based on fine tuning for material decomposition [28], [42]. However, in this work we follow a Sim2Real transfer approach where the knowledge that one learns from synthetic data is transferred to a real-world scenario [49], [50].…”

Experimental study of photon-counting CT neural network material decomposition under conditions of pulse pileup

“…In previous work, NN material decomposition was investigated under low tube current settings with negligible pileup effects. 13,14 This previous work demonstrated that the NN can learn to compensate for the potential bias due to flux-independent spectral detector effects, such as charge sharing.…”

Interweaving Network: A Novel Monochromatic Image Synthesis Method for a Photon-Counting Detector CT System

“…Different from Touch, the output of neural network is directly set as decomposition coefficient in projection domain. They also proposed some transfer learning strategies such as from simulation data to experimental data and from aggregated pixels to individual pixel [20]. In addition to these sinogram domain deep learning-based methods, there are also some methods working in image domain [21], [22].…”