Environmental decentralization (ED), as an institutional factor, impacts the effectiveness of environmental regulation to achieve green productivity (GTFP). Based on panel data of 30 Chinese provinces from 2001 to 2015, this paper assessed GTFP, whose value is higher than 1. A two-step generalized method of moments (GMM) was employed to test the effects of environmental regulations (environmental protection investment (ENV) and pollutant discharge fees (PDF)) on GTFP with or without being influenced by ED. Without the impact of ED, GTFP is significantly inhibited by ENV, while it is significantly promoted by PDF. Under the influences of environmental decentralization from the central to the local authorities (TED), ENV has insignificantly negative effects on GTFP; contrarily, PDF have positive effects on GTFP. As for moderating effects of environmental decentralization at different administrative levels within a province, the degree of provincial environmental decentralization (PTED) should be decreased, while the degrees of prefectural ED (UTED) and county-level ED (CTED) should be increased. Generally, rationally allocating environmental management power among various administrative levels in a province increases the effectiveness of PDF to achieve green productivity while decreases the negative effects of ENV.