Anti-Müllerian hormone (AMH) has been demonstrated to exhibit an inhibitory effect on the proliferation, invasion, metastasis and drug resistance of ovarian cancer. However, the mechanisms underlying these effects remain unclear. In the present study, 10 µg/ml recombinant human AMH (rhAMH) was administered to human OVCAR3 and OVCAR8 epithelial ovarian cancer (EOC) cell lines. Cell proliferation, apoptosis and cell cycle were analyzed. The level of stem cell factor (SCF) was detected using a reverse transcription-quantitative polymerase chain reaction and an ELISA, respectively. The exogenous addition of rhAMH significantly reduced the proliferation of OVCAR3 and OVCAR8 cell lines compared with the control group (P<0.01). The apoptosis rate in the rhAMH treated group (48 h) significantly increased compared with in the control group (OVCAR3, P=0.035; OVCAR8, P=0.020). The apoptosis rate increased at 72 h but did not exhibit a significant difference when compared with the 48 h group (OVCAR3, P=0.145; OVCAR8, P=0.296). The percentage of cells in the G phase in the rhAMH treated group (48 h) increased but was not significantly different compared with the control group (OVCAR3, P=0.070; OVCAR8, P=0.051). However, there was a significant difference at 72 h compared with the control group (OVCAR3, P=0.016; OVCAR8, P=0.019). At 48 h, the rhAMH-treated group exhibited a statistically significant inhibition of SCF mRNA expression levels (P=0.008), but no significant difference in the protein expression levels (P=0.101) compared with the control, though a significant inhibition was exhibited at 72 h (mRNA expression levels, P=0.005; protein expression levels, P=0.036). The present study revealed that rhAMH may be able to inhibit the proliferation and induce the apoptosis of EOC cells via G/S-phase cell cycle arrest and the decreased secretion of SCF.