Breast cancer is one of the most frequently diagnosed cancer in women and is the major cause of most cancer-related deaths. We previously reported that Brachyury, as a sensitive and specific marker, has been verified to involve in the process of carcinogenesis and progression of breast cancer, but the mechanism by which Brachyury promotes breast cancer cells proliferation and migration still remains less clear. In this study, we identified that Brachyury was markedly increased in breast cancer compared with the adjacent tissues. We have also shown that Brachyury knockdown could decrease the proliferation and migration capability in breast cancer cells both in vitro and in vivo. Finally, we found an important transcriptional factor, E2F3, which is a direct downstream target gene of Brachyury by chromatin immunoprecipitation (ChIP) analysis. Knockdown of E2F3 also decreased breast cancer cell proliferation and migration. Taken together, we reported that Brachyury may act as an oncogenic role in the progression of breast cancer by positively-regulating E2F3 expression.