Repair of articular cartilage defects is a challenging aspect of clinical treatment. Kartogenin (KGN), a small molecular compound, can induce the differentiation of bone marrow-derived mesenchymal stem cells (BMSCs) into chondrocytes. Here, we constructed a scaffold based on chondrocyte extracellular matrix (CECM) and poly(lactic-co-glycolic acid) (PLGA) microspheres (MP), which can slowly release KGN, thus enhancing its efficiency. Cell adhesion, live/dead staining, and CCK-8 results indicated that the PLGA(KGN)/CECM scaffold exhibited good biocompatibility. Histological staining and quantitative analysis demonstrated the ability of the PLGA(KGN)/CECM composite scaffold to promote the differentiation of BMSCs. Macroscopic observations, histological tests, and specific marker analysis showed that the regenerated tissues possessed characteristics similar to those of normal hyaline cartilage in a rabbit model. Use of the PLGA(KGN)/CECM scaffold may mimic the regenerative microenvironment, thereby promoting chondrogenic differentiation of BMSCs in vitro and in vivo. Therefore, this innovative composite scaffold may represent a promising approach for acellular cartilage tissue engineering.