Borderline ovarian tumor (BOT) accounts for 15–20% of all epithelial ovarian tumors. Concerns have arisen about the clinical and prognostic implications of BOT with exophytic growth patterns. We retrospectively reviewed all cases of BOT patients surgically treated from 2015 to 2020. Patients were divided into an endophytic pattern (with intracystic tumor growth and intact ovarian capsule) and an exophytic pattern (with tumor growth outside the ovarian capsule) group. Among the 254 patients recruited, 229 met the inclusion criteria, and of these, 169 (73.8%) belonged to the endophytic group. The endophytic group showed more commonly an early FIGO stage than the exophytic group (100.0% vs. 66.7%, p < 0.001). Furthermore, tumor cells in peritoneal washing (20.0% vs. 0.6%, p < 0.001), elevated Ca125 levels (51.7% vs. 31.4%, p = 0.003), peritoneal implants (0 vs. 18.3%, p < 0.001), and invasive peritoneal implants (0 vs. 5%, p = 0.003) were more frequently observed in the exophytic group. The survival analysis showed 15 (6.6%) total recurrences, 9 (5.3%) in the endophytic and 6 (10.0%) patients in the exophytic group (p = 0.213). At multivariable analysis, age (p = 0.001), FIGO stage (p = 0.002), fertility-sparing surgery (p = 0.001), invasive implants (p = 0.042), and tumor spillage (p = 0.031) appeared significantly associated with recurrence. Endophytic and exophytic patterns in borderline ovarian tumors show superimposable recurrence rates and disease-free survival.