It is well established that innervation is one of the updated hallmarks of cancer and that psychological stress promotes the initiation and progression of cancer. The breast tumor environment includes not only fibroblasts, adipocytes, endothelial cells, and lymphocytes but also neurons, which is increasingly discovered important in breast cancer progression. Peripheral nerves, especially sympathetic, parasympathetic, and sensory nerves, have been reported to play important but different roles in breast cancer. However, their roles in the breast cancer progression and treatment are still controversial. In addition, the brain is one of the favorite sites of breast cancer metastasis. In this review, we first summarize the innervation of breast cancer and its mechanism in regulating cancer growth and metastasis. Next, we summarize the neural-related molecular markers in breast cancer diagnosis and treatment. In addition, we review drugs and emerging technologies used to block the interactions between nerves and breast cancer. Finally, we discuss future research directions in this field. In conclusion, the further research in breast cancer and its interactions with innervated neurons or neurotransmitters is promising in the clinical management of breast cancer.