To understand the effects of source-sink relationships on rice yield response to elevated CO2 levels (eCO2), we conducted a field study using a popular japonica cultivar grown in a free-air CO2 enrichment environment in 2017–2018. The source-sink ratio of rice was set artificially via source-sink treatments (SSTs) at the heading stage. Five SSTs were performed in 2017 (EXP1): cutting off the flag leaf (LC1) and the top three functional leaves (LC3), removing one branch in every three branches of a panicle (SR1/3) and one branch in every two branches of a panicle (SR1/2), and the control (CK) without any leaf cutting or spikelet removal. The eCO2 significantly increased grain yield by 15.7% on average over all treatments; it significantly increased grain yield of CK, LC1, LC3, SR1/3, and SR1/2 crops by 13.9, 18.1, 25.3, 12.0, and 10.9%, respectively. The yield response to eCO2 was associated with a significant increase of panicle number and fully-filled grain percentage (FGP), and the response of crops under different SSTs was significantly positively correlated with FGP and the average grain weight of the seeds. Two SSTs (CK and LC3) were performed in 2018 (EXP2), which confirmed that the yield response of LC3 crops (25.1%) to eCO2 was significantly higher than that of CK (15.9%). Among the different grain positions, yield response to eCO2 of grains attached to the lower secondary rachis was greater than that of grains attached to the upper primary rachis. Reducing the source-sink ratio via leaf-cutting enhanced the net photosynthetic rate response of the remaining leaves to eCO2 and increased the grain filling ability. Conversely, spikelet removal increased the non-structural carbohydrate (NSC) content of the stem, causing feedback inhibition and photosynthetic down-regulation. This study suggests that reducing the source-sink ratio by adopting appropriate management measures can increase the response of rice to eCO2.