The objective of this study was to compare the demographic characteristics, clinicopathological factors and survival outcomes between infiltrating lobular carcinoma (ILC) and infiltrating ductal carcinoma (IDC) using our single-center database. Methods: Seventeen thousand two hundred and three breast cancer patients were treated at Fudan University Shanghai Cancer Center (FUSCC) from January 2000 to December 2017. We identified 365 cases with ILC and 16,838 cases with IDC. The Pearson chi-square test was used to compare tumor characteristics, and the Kaplan-Meier methods were used to perform the survival analysis. Results: ILC had some distinctive characteristics from IDC such as older age (ranged from 61 to 80: ILC 26.8% vs IDC 19.9%, P < 0.001; over 80: ILC 1.6% vs IDC 0.8%, P < 0.001), larger tumor size (ranged from 2 to 5: ILC 45.2% vs IDC 37.1%, P = 0.011), much more hormone receptor expression (ILC 92.9% vs IDC 73.0%, P < 0.001), extremely less HER-2 expression (ILC 7.1% vs IDC 25.9%, P < 0.001). The overall survival and disease-free survival of ILC were worse than IDC (5-year OS, ILC 93.6% vs IDC 94.5%, P < 0.001; 5-year DFS, ILC 88.5% vs IDC 91.6%, P = 0.008). It was worth noting that the ILC patients had a worse overall survival than IDC patients after our propensity score matching study (P = 0.037). The univariate analysis concluded that positive HR (hormone receptor), high expression of Ki-67 and higher pathologic tumor stage were poor prognostic markers of ILC. Multivariate analysis demonstrated that tumor stage was a poor prognostic marker after adjustment for the effects of the above three factors. The most common primary site of metastasis was bone, but the proportion in the ILC group was much higher than that in the IDC group (56.25% vs 36.40%, P = 0.003). Conclusion: Compared with IDC, ILC survived worse and was more prone to bone metastasis. Therefore, a comprehensive understanding of ILC and specific treatments are needed for further research.