Pecan [Carya illinoinensis (Wangenh.) K. Koch] is a high-value fruit tree with a long juvenile period. The fruiting process of pecan seedlings can be largely accelerated through grafting. As non-coding small RNAs, plant miRNAs participate in various biological processes through negative regulation of gene expression. To reveal the roles of miRNAs in the graft union development of pecan, four small RNA libraries were constructed from the graft union at days 0, 8, 15, and 30 after grafting. A total of 47 conserved miRNAs belonging to 31 families and 39 novel miRNAs were identified. For identified miRNAs, 584 target genes were bioinformatically predicted, and 266 of them were annotated; 29 miRNAs (including 16 conserved and 13 novel miRNAs) were differentially expressed during the graft process. The expression profiles of 12 miRNA were further validated by quantitative reverse transcription PCR (qRT-PCR). In addition, qRT-PCR revealed that the expression levels of 3 target genes were negatively correlated with their corresponding miRNAs. We found that miRS26 might be involved in callus formation; miR156, miR160, miR164, miR166, and miRS10 might be associated with vascular bundle formation. These results indicate that the miRNA-mediated gene regulations play important roles in the graft union development of pecan.