Background Neuroendocrine carcinoma (NEC) of the breast is a very rare malignant tumor whose clinic pathological features and prognosis are not well defined. The aim of this study was to compare the clinic pathological features and clinical outcome between breast neuroendocrine carcinoma and breast invasive ductal carcinoma (IDC) and to know the independent prognostic risk factors of NEC.Method Patients with breast NEC and breast IDC were identified through the Surveillance, Epidemiology, and End Results (SEER) database from 2010 to 2015. Then we compared the clinic pathological features and outcome between the two cohorts. Propensity score matching (PSM) was performed to balance the effects of baseline clinic pathological differences. Potential prognostic factors of breast NEC were identified by the univariate and multivariate Cox analysis.Results In total, we identified 208 patients with breast NEC and 275,878 cases with breast IDC.Compared with IDC, NEC was significantly correlated with higher stage, larger tumor size, more distant transfers and no population of HER2 enriched subtype. There were significant differences on the choice of treatment therapy between NEC and IDC (p=0.000). More patients chose not to undergo surgery (35.1% vs. 9.0%), and fewer patients chose breast conserving surgery than IDC patients (32.2% vs. 51.1%). Compared with IDC patients, NEC patients were more likely to choose chemotherapy (p=0.039) and more conservative in radiotherapy (p=0.000). The 66-month overall survival (OS) rate in NEC and IDC was 44.5% and 80.2% before PSM and was 49.8% and 71.4% after PSM, respectively. The 66-month breast cancer specific survival (BCSS) rate in NEC and IDC was 56.3% and 88.8% before PSM and was 54.1% and 80.5% after PSM, respectively. Finally, we identified independent prognostic factors, age, stage, pathological type, distant metastasis, chemotherapy, and molecular typing. Conclusions Breast NEC has distinct clinic pathological features and a significantly worse clinical outcome than common IDC.