Cadmium is a toxic metal widely found in workplaces and plant soil because of extensive industrialization. Wheat is an important source of food generated from plant soil. The different responses of wheat against different omic levels of cadmium have been observed and widely studied worldwide. With the development of high-throughput sequencing, micro-level biological research has extended to the microRNA level. In this study, high-cadmium-accumulating wheat cultivars (Annong9267) and low-cadmium-accumulating wheat cultivars (Qian 102032) were used as experimental models. The two cultivars were treated by Cd for 2 h to explore the microRNA profiles in root and leaf tissues through small RNA sequencing. Important small RNAs, such as tae-miR9663-5p and tae-miR6201, and potential small RNA-mediated mechanisms associated with cadmium accumulation were identified by summarizing specific microRNA profiling patterns and their respective target genes. At the wheat roots and leaves, differentially expressed small RNAs related to cadmium accumulation in different plant tissues (roots or leaves) were identified, and functional enrichment analyses on target genes of differentially expressed miRNAs in low- and high-cadmium-accumulating wheat cultivars in different plant tissues (roots or leaves) obtained some known mature miRNAs and new miRNAs. The identified miRNA will be regarded as a potential screening biomarker for low-cadmium-accumulating wheat.