Using radiogenomics methods, the differences between tumor imaging data and genetic data in Chinese and Western breast cancer (BC) patients were analyzed, and the correlation between phenotypic data and genetic data was explored. In this paper, we analyzed BC patients’ image characteristics and transcriptome data separately, then correlated the magnetic resonance imaging (MRI) phenotype with the transcriptome data through a computational method to develop a radiogenomics feature. The data was fed into the designed random forest (RF) model, which used the area under the receiver operating curve (AUC) as the evaluation index. Next, we analyzed the hub genes in the differentially expressed genes (DEGs) and obtained seven hub genes, which may cause Chinese and Western BC patients to behave differently in the clinic. We demonstrated that combining relevant genetic data and imaging features could better classify Chinese and Western patients than using genes or imaging characteristics alone. The AUC values of 0.74, 0.81, and 0.95 were obtained separately using the image characteristics, DEGs, and radiogenomics features. We screened SYT4, GABRG2, CHGA, SLC6A17, NEUROG2, COL2A1, and MATN4 and found that these genes were positively or negatively correlated with certain imaging characteristics. In addition, we found that the SLC6A17, NEUROG2, CHGA, and MATN4 genes were associated with clinical features.