PurposeTo develop a deep neural network system for the automatic segmentation and risk stratification prediction of gastrointestinal stromal tumors (GISTs).MethodsA total of 980 ultrasound (US) images from 245 GIST patients were retrospectively collected. These images were randomly divided (6:2:2) into a training set, a validation set, and an internal test set. Additionally, 188 US images from 47 prospective GIST patients were collected to evaluate the segmentation and diagnostic performance of the model. Five deep learning‐based segmentation networks, namely, UNet, FCN, DeepLabV3+, Swin Transformer, and SegNeXt, were employed, along with the ResNet 18 classification network, to select the most suitable network combination. The performance of the segmentation models was evaluated using metrics such as the intersection over union (IoU), Dice similarity coefficient (DSC), recall, and precision. The classification performance was assessed based on accuracy and the area under the receiver operating characteristic curve (AUROC).ResultsAmong the compared models, SegNeXt‐ResNet18 exhibited the best segmentation and classification performance. On the internal test set, the proposed model achieved IoU, DSC, precision, and recall values of 82.1, 90.2, 91.7, and 88.8%, respectively. The accuracy and AUC for GIST risk prediction were 87.4 and 92.0%, respectively. On the external test set, the segmentation models exhibited IoU, DSC, precision, and recall values of 81.0, 89.5, 92.8, and 86.4%, respectively. The accuracy and AUC for GIST risk prediction were 86.7 and 92.5%, respectively.ConclusionThis two‐stage SegNeXt‐ResNet18 model achieves automatic segmentation and risk stratification prediction for GISTs and demonstrates excellent segmentation and classification performance.