Polycaprolactone (PCL) has recently received significant attention due to its mechanical strength, low immunogenicity, elasticity, and biodegradability. Therefore, it is perfectly suitable for cartilage tissue engineering. PCL is relatively hydrophobic in nature, so its hydrophilicity needs to be enhanced before its use in scaffolding. In our study, first, we aimed to improve the hydrophilicity properties after the network of the bilayer scaffold was formed by electrospinning. Electrospun bilayer PCL scaffolds were treated with ozone and further loaded with transforming growth factor-beta 3 (TGFβ3). In vitro studies were performed to determine the rabbit muscle-derived stem cells’ (rMDSCs) potential to differentiate into chondrocytes after the cells were seeded onto the scaffolds. Statistically significant results indicated that ozonated (O) scaffolds create a better environment for rMDSCs because collagen-II (Coll2) concentrations at day 21 were higher than non-ozonated (NO) scaffolds. In in vivo studies, we aimed to determine the cartilage regeneration outcomes by macroscopical and microscopical/histological evaluations at 3- and 6-month time-points. The Oswestry Arthroscopy Score (OAS) was the highest at both mentioned time-points using the scaffold loaded with TGFβ3 and rMDSCs. Evaluation of cartilage electromechanical quantitative parameters (QPs) showed significantly better results in cell-treated scaffolds at both 3 and 6 months. Safranin O staining indicated similar results as in macroscopical evaluations—cell-treated scaffolds revealed greater staining with safranin, although an empty defect also showed better results than non-cell-treated scaffolds. The scaffold with chondrocytes represented the best score when the scaffolds were evaluated with the Mankin histological grading scale. However, as in previous in vivo evaluations, cell-treated scaffolds showed better results than non-cell-treated scaffolds. In conclusion, we have investigated that an ozone-treated scaffold containing TGFβ3 with rMDSC is a proper combination and could be a promising scaffold for cartilage regeneration.