Primary neuroendocrine carcinoma of the breast (NECB) is characterized with heterogeneity, rarity, and poor differentiation, which is probably an underestimated subtype of breast cancer, including small cell NECs and large cell NECs. The diagnostic criteria for NECB have been constantly updated as the disease changes and the understanding increases. According to the latest WHO Classification, primary neuroendocrine neoplasm (NEN) of the breast consists of well-differentiated neuroendocrine tumors (NET), extremely aggressive neuroendocrine carcinomas (NEC) as well as invasive breast cancers of no special type (IBCs-NST) with neuroendocrine differentiation. The accurate diagnosis of NECB remains a challenge for its low incidence, which needs multi-disciplinary methods. For the rarity of the disease, there is a lack of large samples and prospective clinical research. For these invasive tumors, there are no standardized therapeutic guidelines or norms, and the treatment often refers to nonspecific breast cancer. In addition, the prognosis of such patients remains unknown. In 2003, the World Health Organization (WHO) listed NECB as an independent entity for the first time, while few features of NECB were clarified. In this review, it presents the WHO Classification, clinicopathologic characteristics, diagnosis, treatment, and prognosis of these patients. In addition, it summarizes the latest studies on molecular features of NECB, aiming to provide new therapeutic perspectives for the disease.