MicroRNAs (miRNAs), a class of noncoding small RNAs (sRNAs), are widely involved in the response to high temperature (HT) stress at both the seedling and flowering stages. To dissect the roles of miRNAs in regulating male fertility in soybean cytoplasmic male sterility (CMS)-based F1 under HT, sRNA sequencing was performed using flower buds from HT-tolerant and HT-sensitive CMS-based F1 combinations (NF1 and YF1, respectively). A total of 554 known miRNAs, 59 new members of known miRNAs, 712 novel miRNAs, and 1145 target genes of 580 differentially expressed miRNAs (DEMs) were identified under normal temperature and HT conditions. Further integrated analysis of sRNA and transcriptome sequencing found that 21 DEMs and 15 differentially expressed target genes, such as gma-miR397a/Laccase 2, gma-miR399a/Inorganic phosphate transporter 1-4, and gma-miR4413a/PPR proteins, mitochondrial-like, were negatively regulated under HT stress. Furthermore, all members of the gma-miR156 family were suppressed by HT stress in both NF1 and YF1, but were highly expressed in YF1 under HT condition. The negative correlation between gma-miR156b and its target gene squamosa promoter-binding protein-like 2b was confirmed by expression analysis, and overexpression of gma-miR156b in Arabidopsis led to male sterility under HT stress. With these results, we proposed that miRNAs play an important role in the regulation of male fertility stability in soybean CMS-based F1 under HT stress.