Medium‐ or late‐maturing varieties of rice (Oryza sativa L.) that are planted in agroecosystems in the middle and lower reaches of the Yangtze Valley of China always encounter cold stress during the late grain‐filling stage. Generally, grain yield and yield components are adversely affected under such circumstances. To alleviate these negative effects, salicylic acid (SA; i.e., SA0 [control, 0 mg L−1], SA100 [100 mg L−1], SA200 [200 mg L−1], SA300 [300 mg L−1], and SA400 [400 mg L−1]) was exogenously sprayed on plants of three rice cultivars at full heading in the field at the experimental base of China National Rice Research Institute (CNRRI) in Fuyang City, China (30°03′ N, 119°57′ E, 11 m asl). The results indicated that the mean grain‐filling rate and grain weight of the inferior grains were significantly increased under the SA200 (200 mg L−1) treatment and correspondingly contributed to a grain yield increase of 5.0 to 10.1% in comparison with the control. Further analysis showed that the SA200 treatment significantly increased the activity of the starch branching enzyme in inferior grains over the entire grain‐filling stage. Additionally, the photosynthetic rate, chlorophyll content, and soluble protein content of the flag leaf, which precisely reflected the senescence status, were significantly increased under the SA200 treatment, especially during late maturity under mild cold stress. We speculated that improved physiological changes during the late grain‐filling stage account for the increased grain weight and grain yield. Therefore, application of SA was a promising method of alleviating the adverse effects that were induced by mild cold stress on the grain yield of rice in the middle and lower reaches of the Yangtze Valley of China.