A convolutional neural network with self-attention for fully automated metabolic tumor volume delineation of head and neck cancer in $$[^{18}$$F]FDG PET/CT

Abstract

Purpose PET-derived metabolic tumor volume (MTV) and total lesion glycolysis of the primary tumor are known to be prognostic of clinical outcome in head and neck cancer (HNC). Including evaluation of lymph node metastases can further increase the prognostic value of PET but accurate manual delineation and classification of all lesions is time-consuming and prone to interobserver variability. Our goal, therefore, was development and evaluation of an automated tool for MTV delineation/classification of primary tumor and lymph node metastases in PET/CT investigations of HNC patients. Methods Automated lesion delineation was performed with a residual 3D U-Net convolutional neural network (CNN) incorporating a multi-head self-attention block. 698 $$[^{18}$$ [ 18 F]FDG PET/CT scans from 3 different sites and 5 public databases were used for network training and testing. An external dataset of 181 $$[^{18}$$ [ 18 F]FDG PET/CT scans from 2 additional sites was employed to assess the generalizability of the network. In these data, primary tumor and lymph node (LN) metastases were interactively delineated and labeled by two experienced physicians. Performance of the trained network models was assessed by 5-fold cross-validation in the main dataset and by pooling results from the 5 developed models in the external dataset. The Dice similarity coefficient (DSC) for individual delineation tasks and the primary tumor/metastasis classification accuracy were used as evaluation metrics. Additionally, a survival analysis using univariate Cox regression was performed comparing achieved group separation for manual and automated delineation, respectively. Results In the cross-validation experiment, delineation of all malignant lesions with the trained U-Net models achieves DSC of 0.885, 0.805, and 0.870 for primary tumor, LN metastases, and the union of both, respectively. In external testing, the DSC reaches 0.850, 0.724, and 0.823 for primary tumor, LN metastases, and the union of both, respectively. The voxel classification accuracy was 98.0% and 97.9% in cross-validation and external data, respectively. Univariate Cox analysis in the cross-validation and the external testing reveals that manually and automatically derived total MTVs are both highly prognostic with respect to overall survival, yielding essentially identical hazard ratios (HR) ($$\text {HR}_{\text {man}}=1.9$$ HR man = 1.9 ; $$p<0.001$$ p < 0.001 vs. $$\text {HR}_{\text {cnn}}=1.8$$ HR cnn = 1.8 ; $$p<0.001$$ p < 0.001 in cross-validation and $$\text {HR}_{\text {man}}=1.8$$ HR man = 1.8 ; $$p=0.011$$ p = 0.011 vs. $$\text {HR}_{\text {cnn}}=1.9$$ HR cnn = 1.9 ; $$p=0.004$$ p = 0.004 in external testing). Conclusion To the best of our knowledge, this work presents the first CNN model for successful MTV delineation and lesion classification in HNC. In the vast majority of patients, the network performs satisfactory delineation and classification of primary tumor and lymph node metastases and only rarely requires more than minimal manual correction. It is thus able to massively facilitate study data evaluation in large patient groups and also does have clear potential for supervised clinical application.

Transformer-CNN hybrid network for improving PET time of flight prediction

“…To address these limitations, a self-attention mechanism capturing long-range dependencies and global information in waveforms needs to be introduced. In the field of medical imaging, some methods have incorporated self-attention mechanism into CNNs (Wu et al 2019, Nikulin et al 2023. However, the handling of one-dimensional signals in detector waveforms is different, as temporal information with temporal dependencies needs to be considered.…”