Lake sediment substantially accumulates nutrients, while little is known regarding the conditions under which it tends to be a sink or source of phosphorus. It is postulated that the above functions were largely dependent on trophic state. To test this hypothesis, composition, abundance, and size-spectrum of phytoplankton were studied in a Chinese shallow lake (Lake Donghu), together with concentrations and sorption behaviors of phosphorus in water column and sediment. Relationships among these variables were also examined by structural equation model. In the basins with the lowest phosphorus concentration, sediment was a phosphorus source as judged by equilibrium phosphorus concentration, directly affecting the abundance of phytoplankton with smaller size. Contrastingly, in the basins with the highest phosphorus concentration, sediment tended to uptake phosphorus, companied by the lowest ratio of nitrogen to phosphorus, diversity, and evenness of phytoplankton with the smaller size one dominating. Oscillatoria sp. was directly affected by ammonia and phosphorus. Moreover, with a balanced exchange of phosphorus between sediment and water column, phytoplankton increased markedly with the highest diversity and evenness in the basins having intermediate phosphorus concentration. Overall, trophic state modulated sediment functions to supply nutrient and was in turn greatly affected by the sediment in shallow lakes.