BackgroundNicotine has negative effects on tissue repair, little research concerns its effect on the cartilage repair of tissue engineering stem cells. The present study aimed to investigate the effects of nicotine on the bone marrow-derived mesenchymal stem cells’ (BMSCs) chondrogenic repair function of cartilage defects and explored the molecular mechanism.MethodsA cartilage defect model of rat was repaired by BMSC transplantation, and treated with nicotine or saline at 2.0 mg/kg/d in 12 weeks. Nicotine’s effect on chondrogenic differentiation was studied by exposing BMSCs to nicotine at 0.1, 1, 10, and 100 μM, and methyllycaconitine (MLA), which is a selective α7-nicotinic acetylcholine receptor (nAChR) inhibitor and si-RNA of nuclear factor of activated T cells 2 (NFATc2), were used to verify the molecular mechanism of nicotine’s effect.ResultsData showed that nicotine inhibited cartilage repair function by suppressing SRY-type high-mobility group box 9 (Sox9) in regenerated tissues. Further in vitro study demonstrated that nicotine enhanced intracellular Ca2+ and activity of calcineurin (CaN) through α7-nAChR, increased the nucleic expressions of NFATc2 and the bindings to SOX9 promoter, and thus reduced the acetylation of H3K9 and H3K14 in SOX9 promoter.ConclusionsFindings from this study demonstrated that nicotine suppressed the chondrogenic differentiation of BMSCs in vivo and in vitro, which offers insight into the risk assessment of cartilage defect repair in a nicotine exposure population and its therapeutic target.Electronic supplementary materialThe online version of this article (10.1186/s13287-018-0853-x) contains supplementary material, which is available to authorized users.