Background: Artificial intelligence (AI) based radiotherapy treatment planning tools have gained interest in automating the treatment planning process. It is essential to understand their overall robustness in various clinical scenarios. This is an existing gap between many AI based tools and their actual clinical deployment.This study works to fill the gap for AI based treatment planning by investigating a clinical robustness assessment (CRA) tool for the AI based planning methods using a phantom simulation approach.Methods: A cylindrical phantom was created in the treatment planning system (TPS) with the axial dimension of 30 cm by 18 cm. Key structures involved in pancreas stereotactic body radiation therapy (SBRT) including PTV25, PTV33, C-Loop, stomach, bowel and liver were created within the phantom.Several simulation scenarios were created to mimic multiple scenarios of anatomical changes, including displacement, expansion, rotation and combination of three. The goal of treatment planning was to deliver 25 Gy to PTV25 and 33 Gy to PTV33 in 5 fractions in simultaneous integral boost (SIB) manner while limiting luminal organ-at-risk (OAR) max dose to be under 29 Gy. A previously developed deep learning based AI treatment planning tool for pancreas SBRT was identified as the validation object. For each scenario, the anatomy information was fed into the AI tool and the final fluence map associated to the plan was generated, which was subsequently sent to TPS for leaf sequencing and dose calculation. The final auto plan's quality was analyzed against the treatment planning constraint. The final plans' quality was further analyzed to evaluate potential correlation with anatomical changes using the Manhattan plot.Results: A total of 32 scenarios were simulated in this study. For all scenarios, the mean PTV25 V25Gy of the AI based auto plans was 96.7% while mean PTV33 V33Gy was 82.2%. Large variation (16.3%) in PTV33 V33Gy was observed due to anatomical variations, a.k.a. proximity of luminal structure to PTV33.