Cultivation of green manure (GM) crops in intensive cropping systems is important for enhancing crop productivity through soil quality improvement. We investigated yield sustainability, nutrient stocks, nutrient balances and enzyme activities affected by different long-term (1982–2016) green manure rotations in acidic paddy soil in a double-rice cropping system. We selected four treatments from a long-term experiment, including (1) rice-rice-winter fallow as a control treatment (R-R-F), (2) rice-rice-milkvetch (R-R-M), (3) rice-rice-rapeseed (R-R-R), and (4) rice-rice-ryegrass (R-R-G). The results showed that different GM rotations increased grain yield and the sustainable yield index compared with those of the R-R-F treatment. Compared with those of R-R-F, the average grain yield of early rice in R-R-M, R-R-R, and R-R-G increased by 45%, 29%, and 27%, respectively and that of late rice increased by 46%, 28%, and 26%, respectively. Over the years, grain yield increased in all treatments except R-R-F. Green manure also improved the soil chemical properties (SOM and total and available N and P), except soil pH, compared to those of the control treatment. During the 1983–1990 cultivation period, the soil pH of the R-R-M treatment was lower than that of the R-R-F treatment. The addition of green manure did not mitigate the soil acidification caused by the use of inorganic fertilizers. The soil organic matter (SOM), total nitrogen (TN) and total phosphorus (TP) contents and stocks of C, N and P increased over the years. Furthermore, GM significantly increased phosphatase and urease activities and decreased the apparent N and P balances compared with those in the winter fallow treatment. Variance partitioning analysis revealed that soil properties, cropping systems, and climatic factors significantly influenced annual grain yield. Aggregated boosted tree (ABT) analysis quantified the relative influences of the different soil properties on annual grain yield and showed that the relative influences of TN content, SOM, pH, and TP content on annual crop yield were 27.8%, 25.7%, 22.9%, and 20.7%, respectively. In conclusion, GM rotation is beneficial for sustaining high crop yields by improving soil biochemical properties and reducing N and P balances in acidic soil under double- rice cropping systems.