SUMMARY Tiller number is one of the most important agronomic traits that determine rice (Oryza sativa) yield. Active growth of tiller bud (TB) requires high amount of mineral nutrients; however, the mechanism underlying the distribution of mineral nutrients to TB with low transpiration is unknown. Here, we found that the distribution of Zn to TB is mediated by OsZIP4, one of the ZIP (ZRT, IRT‐like protein) family members. The expression of OsZIP4 was highly detected in TB and nodes, and was induced by Zn deficiency. Immunostaining analysis revealed that OsZIP4 was mainly expressed in phloem of diffuse vascular bundles in the nodes and the axillary meristem. The mutation of OsZIP4 did not affect the total Zn uptake, but altered Zn distribution; less Zn was delivered to TB and new leaf, but more Zn was retained in the basal stems at the vegetative growth stage. Bioimaging analysis showed that the mutant aberrantly accumulated Zn in enlarged and transit vascular bundles of the basal node, whereas in wild‐type high accumulation of Zn was observed in the meristem part. At the reproductive stage, mutation of OsZIP4 resulted in delayed panicle development, which is associated with decreased Zn distribution to the panicles. Collectively, OsZIP4 is involved in transporting Zn to the phloem of diffuse vascular bundles in the nodes for subsequent distribution to TBs and other developing tissues. It also plays a role in transporting Zn to meristem cells in the TBs.