Cypripedium japonicum is an endangered terrestrial orchid species with high ornamental and medicinal value. As global warming continues to intensify, the survival of C. japonicum will be further challenged. Understanding the impact of climate change on its potential distribution is of great significance to conserve this species. In this study, we established an ensemble species distribution model based on occurrence records of C. japonicum and 13 environmental variables to predict its potential distribution under current and future climatic conditions. The results show that the true skill statistic (TSS), Cohen’s kappa statistic (Kappa), and the area under the receiver operating characteristic curve (AUC) values of the ensemble model were 0.968, 0.906, and 0.995, respectively, providing more robust predictions. The key environmental variables affecting the distribution of C. japonicum were the precipitation in the warmest quarter (Bio18) and the mean temperature in the driest quarter (Bio9). Under future climatic conditions, the total suitable habitat of C. japonicum will increase slightly and tend to migrate northwestward, but the highly suitable areas will be severely lost. By 2070, the loss of its highly suitable habitat area will reach 57.69–72.24% under representative concentration pathway (RCP) 4.5 and 8.5 respectively, and the highly suitable habitats in Zhejiang and Anhui will almost disappear. It is noteworthy that the highly suitable habitat of C. japonicum has never crossed the Qinba mountainous area during the migration process of the suitable habitat to the northwest. Meanwhile, as the best-preserved area of highly suitable habitat for C. japonicum in the future, the Qinba mountainous area is of great significance to protect the wild germplasm resources of C. japonicum. In addition, we found that most of the changes predicted for 2070 will already be seen in 2050; the problem of climate change may be more urgent than it is believed.