Accurate material performance parameters are the prerequisite for conducting composite material structural analysis and design. However, the complex multiscale structure of ceramic matrix composites (CMCs) makes it extremely difficult to accurately obtain their mechanical performance parameters. To address this issue, a CMC micro-scale constituents (fiber bundles and matrix) elastic parameter inversion method was proposed based on the integration of macro–micro finite element models. This model was established based on the μCT scan data of a plain-woven CMC tensile specimen using the chemical vapor infiltration (CVI) process, which could reflect the real microstructure and surface morphology characteristics of the material. A BP neural network was used to predict the multiscale stiffness, considering the influence of the porous structure on the macroscopic stiffness of the material. The inversion process of the constituent elastic parameters was established using the trust-region algorithm combined with an improved error function. The inversion results showed that this method could accurately invert the CMC constituent elastic parameters with excellent robustness and anti-noise performance. Under four different degrees of deviation in the initial iteration conditions, the inversion error of all parameters was within 1%, and the maximum inversion error was only 2.16% under a 10% high noise level.