BackgroundTriple-negative breast cancer (TNBC) has a high risk of relapse and there are few chemotherapy options. Although 5-hydroxytryptamine (5-HT, serotonin) signaling pathways have been suggested as potential targets for anti-cancer drug development, the mechanism responsible for the action of 5-HT in TNBC remains unknown.MethodsQuantitative real-time polymerase chain reaction (qRT-PCR) and Western blotting were used to measure mRNA and protein levels, respectively. Cell proliferation was measured using CellTiter 96 Aqueous One Solution. siRNA transfection was used to assess involvement of genes in cancer invasion, which were identified by Matrigel transwell invasion assay. Levels of 5-HT and vascular endothelial growth factor (VEGF) were measured using ELISA kits. Chick chorioallantoic membrane (CAM) assay and mouse tumor model were used to investigate the in vivo effects of SB269970, a 5-HT7 receptor antagonist, and BJ-1113, a novel synthetic compound.ResultsTNBC cell lines (MDA-MB-231, HCC-1395, and Hs578T) expressed higher levels of tryptophan hydroxylase 1 (TPH1) than hormone-responsive breast cancer cell lines (MCF-7 and T47D). In MDA-MB-231 cells, 5-HT promoted invasion and proliferation via 5-HT7 receptor, and interestingly, the stimulatory effect of 5-HT on MDA-MB-231 cell invasion was stronger than its effect on proliferation. Likewise, downstream signaling pathways of 5-HT7 differed during invasion and proliferation, that is, Gα-activated cAMP and Gβγ-activated kinase signaling during invasion, and Gβγ-activated PI3K/Akt signaling during proliferation. Also, 5-HT increased the protein expressions of TPH1 and VEGF in MDA-MB-231 cells. These results provide insight of the stimulatory effect of 5-HT on breast cancer progression; 5-HT was found to act more strongly during the first stage of metastasis (during invasion and migration) than during the later proliferative phase after local invasion. Interestingly, these actions of 5-HT were inhibited by BJ-1113, a 6-amino-2,4,5-trimethylpyridin-3-ol analog. BJ-1113 blocked intracellular signaling pathways initiated by 5-HT7 receptor activation, and exhibited anti-proliferative and anti-invasive activities against MDA-MB-231 cells. Furthermore, the inhibitory effect of BJ-1113 against MDA-MB-231 tumor growth was greater than that of SB269970, a 5-HT7 receptor antagonist.Conclusions5-HT7 receptor which mediates 5-HT-induced cancer progression is a potential therapeutic target in TNBC, and BJ-1113 offers a novel scaffold for the development of anti-cancer agents against TNBC.