BackgroundDeep learning–based segmentation algorithms usually required large or multi‐institute data sets to improve the performance and ability of generalization. However, protecting patient privacy is a key concern in the multi‐institutional studies when conventional centralized learning (CL) is used.PurposeTo explores the feasibility of a proposed lesion delineation for stereotactic radiosurgery (SRS) scheme for federated learning (FL), which can solve decentralization and privacy protection concerns.Study TypeRetrospective.Subjects506 and 118 vestibular schwannoma patients aged 15–88 and 22–85 from two institutes, respectively; 1069 and 256 meningioma patients aged 12–91 and 23–85, respectively; 574 and 705 brain metastasis patients aged 26–92 and 28–89, respectively.Field Strength/Sequence5T, spin‐echo, and gradient‐echo.AssessmentThe proposed lesion delineation method was integrated into an FL framework, and CL models were established as the baseline. The effect of image standardization strategies was also explored. The dice coefficient was used to evaluate the segmentation between the predicted delineation and the ground truth, which was manual delineated by neurosurgeons and a neuroradiologist.Statistical TestsThe paired t‐test was applied to compare the mean for the evaluated dice scores (p < 0.05).ResultsFL performed the comparable mean dice coefficient to CL for the testing set of Taipei Veterans General Hospital regardless of standardization and parameter; for the Taichung Veterans General Hospital data, CL significantly (p < 0.05) outperformed FL while using bi‐parameter, but comparable results while using single‐parameter. For the non‐SRS data, FL achieved the comparable applicability to CL with mean dice 0.78 versus 0.78 (without standardization), and outperformed to the baseline models of two institutes.Data ConclusionThe proposed lesion delineation successfully implemented into an FL framework. The FL models were applicable on SRS data of each participating institute, and the FL exhibited comparable mean dice coefficient to CL on non‐SRS dataset. Standardization strategies would be recommended when FL is used.Level of Evidence4Technical EfficacyStage 1