Understanding the role of soil organic matter (SOM) in soil quality and subsequent crop yield and input requirements is useful for agricultural sustainability. SOM is widely considered to affect a wide range of soil properties, however, great uncertainty still remains in identifying the relationships between SOM and crop yield due to the difficulty in separating the effect of SOM from other yield-limiting factors. Based on 543 on-farm experiments, where paired treatments with and without NPK fertilizer were conducted during 2005-2009, we quantified the inherent soil productivity, fertilization effect, and their contribution to rice yield and further evaluated their relationships with SOM contents under a rice cropping system in the Sichuan Basin of China. The inherent soil productivity assessed by rice grain yield under no fertilization (Y-CK) was 5.8 t/ha, on average, and contributed 70% to the 8.3 t/ha of rice yield under NPK fertilization (Y-NPK) while the other 30% was from the fertilization effect (FE). No significant correlation between SOM content and Y-NPK was observed, however, SOM content positively related to Y-CK and its contribution to Y-NPK but negatively to FE and its contribution to Y-NPK, indicating an increased soil contribution but a decreased fertilizer contribution to rice yield with increasing SOM. There were significantly positive relationships between SOM and soil available N, P, and K, indicating the potential contribution of SOM to inherent soil productivity by supplying nutrients from mineralization. As a result, approaches for SOM accumulation are practical to improve the inherent soil productivity and thereafter maintain a high crop productivity with less dependence on chemical fertilizers, while fertilization recommendations need to be adjusted with the temporal and spatial SOM variation.