Learning the Physics of Particle Transport via Transformers

Pastor-Serrano, Oscar; Perkó, Zoltán

doi:10.1609/aaai.v36i11.21466

Cited by 8 publications

(19 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the sequential prediction of dose distributions is an important use case for treatment plan optimization, significantly higher prediction throughput and thereby average time per prediction can be achieved by predicting multiple samples in the one step. The deviation from faster prediction times reported for the DoTA model 17 high-end GPU used by the original authors. Depending on the available hardware in future studies, the achieved prediction times will therefore have to be re-evaluated.…”

Section: Discussionmentioning

confidence: 95%

“…It was recently shown that the transformer architecture is very suitable for dose deposition predictions in the case of the proton beam therapy 17,19 . Transformer models rely on the prediction of sequences to sequences utilizing the so‐called attention mechanism 18 .…”

Section: Methodsmentioning

confidence: 99%

“…The translation of the tissue slice sequence into the dose deposition slice sequence is performed using a so-called transformer encoder. 17,18 The slice-based approach of DoTA has been shown to allow a flexible and accurate prediction in a variety of phantoms. This may render this approach superior to a volume-based one such as a 3D U-Net.A thorough intro-duction of the DoTA model is available in the respective published article.…”

Section: Transformer-based Modelmentioning

confidence: 99%

“…It was recently shown that the transformer architecture is very suitable for dose deposition predictions in the case of the proton beam therapy. 17,19 Transformer models rely on the prediction of sequences to sequences utilizing the so-called attention mechanism. 18 In the case of the model of interest, named DoTA, 17,19 the source sequence comprises density matrix tokens, which were obtained from encoding the phantom tissue density matrix slice by slice using a convolutional encoder.…”

Section: Transformer-based Modelmentioning

confidence: 99%

“…17,19 Transformer models rely on the prediction of sequences to sequences utilizing the so-called attention mechanism. 18 In the case of the model of interest, named DoTA, 17,19 the source sequence comprises density matrix tokens, which were obtained from encoding the phantom tissue density matrix slice by slice using a convolutional encoder. The target sequence comprises energy deposition tokens which are decoded via a convolutional decoder into energy deposition slices.…”

Section: Transformer-based Modelmentioning

confidence: 99%

See 4 more Smart Citations

Small beams, fast predictions: a comparison of machine learning dose prediction models for proton minibeam therapy

et al. 2022

View full text Add to dashboard Cite

Background Dose calculations for novel radiotherapy cancer treatments such as proton minibeam radiation therapy is often done using full Monte Carlo (MC) simulations. As MC simulations can be very time consuming for this kind of application, deep learning models have been considered to accelerate dose estimation in cancer patients. Purpose This work systematically evaluates the dose prediction accuracy, speed and generalization performance of three selected state‐of‐the‐art deep learning models for dose prediction applied to the proton minibeam therapy. The strengths and weaknesses of those models are thoroughly investigated, helping other researchers to decide on a viable algorithm for their own application. Methods The following recently published models are compared: first, a 3D U‐Net model trained as a regression network, second, a 3D U‐Net trained as a generator of a generative adversarial network (GAN) and third, a dose transformer model which interprets the dose prediction as a sequence translation task. These models are trained to emulate the result of MC simulations. The dose depositions of a proton minibeam with a diameter of 800μm and an energy of 20–100 MeV inside a simple head phantom calculated by full Geant4 MC simulations are used as a case study for this comparison. The spatial resolution is 0.5 mm. Special attention is put on the evaluation of the generalization performance of the investigated models. Results Dose predictions with all models are produced in the order of a second on a GPU, the 3D U‐Net models being fastest with an average of 130 ms. An investigated 3D U‐Net regression model is found to show the strongest performance with overall 61.0%±$\%\pm$0.5% of all voxels exhibiting a deviation in energy deposition prediction of less than 3% compared to full MC simulations with no spatial deviation allowed. The 3D U‐Net models are observed to show better generalization performance for target geometry variations, while the transformer‐based model shows better generalization with regard to the proton energy. Conclusions This paper reveals that (1) all studied deep learning models are significantly faster than non‐machine learning approaches predicting the dose in the order of seconds compared to hours for MC, (2) all models provide reasonable accuracy, and (3) the regression‐trained 3D U‐Net provides the most accurate predictions.

show abstract

Section: Discussionmentioning

confidence: 95%

Section: Methodsmentioning

confidence: 99%

Section: Transformer-based Modelmentioning

confidence: 99%

Section: Transformer-based Modelmentioning

confidence: 99%

Section: Transformer-based Modelmentioning

confidence: 99%

See 3 more Smart Citations

Small beams, fast predictions: a comparison of machine learning dose prediction models for proton minibeam therapy

et al. 2022

View full text Add to dashboard Cite

show abstract

TrDosePred: A deep learning dose prediction algorithm based on transformers for head and neck cancer radiotherapy

Wang

Zhang

et al. 2023

J Applied Clin Med Phys

View full text Add to dashboard Cite

Background: Intensity-Modulated Radiation Therapy (IMRT) has been the standard of care for many types of tumors. However, treatment planning for IMRT is a time-consuming and labor-intensive process. Purpose: To alleviate this tedious planning process, a novel deep learning based dose prediction algorithm (TrDosePred) was developed for head and neck cancers. Methods: The proposed TrDosePred, which generated the dose distribution from a contoured CT image, was a U-shape network constructed with a convolutional patch embedding and several local self -attention based transformers. Data augmentation and ensemble approach were used for further improvement. It was trained based on the dataset from Open Knowledge-Based Planning Challenge (OpenKBP).The performance of TrDosePred was evaluated with two mean absolute error (MAE) based scores utilized by OpenKBP challenge (i.e., Dose score and DVH score) and compared to the top three approaches of the challenge. In addition, several state-of -the-art methods were implemented and compared to TrDosePred. Results: The TrDosePred ensemble achieved the dose score of 2.426 Gy and the DVH score of 1.592 Gy on the test dataset, ranking at 3rd and 9th respectively in the leaderboard on CodaLab as of writing. In terms of DVH metrics, on average, the relative MAE against the clinical plans was 2.25% for targets and 2.17% for organs at risk. Conclusions: A transformer-based framework TrDosePred was developed for dose prediction. The results showed a comparable or superior performance as compared to the previous state-of -the-art approaches, demonstrating the potential of transformer to boost the treatment planning procedures.

show abstract

Sub‐second photon dose prediction via transformer neural networks

et al. 2023

Self Cite

View full text Add to dashboard Cite

Background: Fast dose calculation is critical for online and real-time adaptive therapy workflows. While modern physics-based dose algorithms must compromise accuracy to achieve low computation times, deep learning models can potentially perform dose prediction tasks with both high fidelity and speed. Purpose: We present a deep learning algorithm that, exploiting synergies between transformer and convolutional layers, accurately predicts broad photon beam dose distributions in few milliseconds. Methods: The proposed improved Dose Transformer Algorithm (iDoTA) maps arbitrary patient geometries and beam information (in the form of a 3D projected shape resulting from a simple ray tracing calculation) to their corresponding 3D dose distribution. Treating the 3D CT input and dose output volumes as a sequence of 2D slices along the direction of the photon beam, iDoTA solves the dose prediction task as sequence modeling. The proposed model combines a Transformer backbone routing long-range information between all elements in the sequence, with a series of 3D convolutions extracting local features of the data. We train iDoTA on a dataset of 1700 beam dose distributions, using 11 clinical volumetric modulated arc therapy (VMAT) plans (from prostate, lung, and head and neck cancer patients with 194-354 beams per plan) to assess its accuracy and speed. Results: iDoTA predicts individual photon beams in ≈ 50 ms with a high gamma pass rate of 97.72 ± 1.93% (2 mm, 2%). Furthermore, estimating full VMAT dose distributions in 6-12 s, iDoTA achieves state-of -the-art performance with a 99.51 ± 0.66% (2 mm, 2%) pass rate and an average relative dose error of 0.75 ± 0.36%. Conclusions: Offering the millisecond speed prediction per beam angle needed in online and real-time adaptive treatments, iDoTA represents a new state of the art in data-driven photon dose calculation. The proposed model can massively speed-up current photon workflows, reducing calculation times from few minutes to just a few seconds.

show abstract

Learning the Physics of Particle Transport via Transformers

Cited by 8 publications

References 45 publications

Small beams, fast predictions: a comparison of machine learning dose prediction models for proton minibeam therapy

Small beams, fast predictions: a comparison of machine learning dose prediction models for proton minibeam therapy

TrDosePred: A deep learning dose prediction algorithm based on transformers for head and neck cancer radiotherapy

Sub‐second photon dose prediction via transformer neural networks

Contact Info

Product

Resources

About