Background: Interactions between adipocyte and breast cancer (BC) cells have yet to be fully elucidated. Here we investigated the prognostic impact of marginal adipose tissue invasion in both luminal breast cancer (HR+/HER2-) and triple-negative breast cancer (TNBC) (HR-/HER2-). Methods: A total of 735 patients with early-stage invasive BC (1999-2014) were retrospectively registered. Median length of patient follow-up was 8.9 years. Survival curves were calculated using a Kaplan-Meier cumulative survival plot. The prognostic difference between two groups were assessed by the univariate Cox-proportional hazard regression model.Results: Patients with adipose tissue invasion (n = 614) had a significantly poorer prognosis than those without adipose tissue invasion (n = 121) in overall survival (OS) (hazard ratio, 2.1; 95% Confidence interval [CI], 1.1 to 4.0; P = 0.025). While a poorer prognosis was observed in TNBC (n = 137) than in luminal BC patients (n = 496) (hazard ratio, 0.45; 95% CI, 0.30 to 0.68, P < 0.001), this aggressive nature of TNBC was noted in node-positive disease (hazard ratio, 0.3; 95% CI, 0.18 to 0.5, P < 0.001) but not in node-negative disease (hazard ratio, 0.78; 95% CI, 0.39 to 1.55, P = 0.472), and also noted in adipose tissue invasion-positive patients (hazard ratio, 0.4; 95% CI, 0.26 to 0.6, P < 0.001) but not in adipose tissue invasion-negative patients (hazard ratio, 0.73; 95% CI, 0.16 to 3.24, P = 0.675). In addition, although patients suffering from TNBC with adipose tissue invasion had a poorer outcome than those without adipose tissue invasion (hazard ratio, 3.63; 95% CI, 1.11 to 11.84; P = 0.033), the difference was not observed in luminal BC (hazard ratio, 1.75; 95% CI, 0.64 to 4.82; P = 0.277). Conclusions: Adipose tissue invasion was correlated with poor survival in TNBC. Cancer cell invasion into local fat may be a first step on cancer progression and systemic disease in TNBC.