Eutrophication is one of the major ecological problems of our era. It accelerates the growth of aquatic plant and algae, eventually leading to ecological deterioration. Based on a 700-day lab experiment, this paper investigated the contrasting effects of sediment microbial fuel cells (SMFCs) on the removal of macrophyte litter in a macrophyte-dominated area and an algae-dominated area from two bay areas of a shallow eutrophic lake. The results revealed that the removal efficiencies of total organic carbon increased by 14.4% in the macrophyte-dominated area and 7.8% in the algae-dominated area. Moreover, it was found that sediment samples from the macrophyte-dominated area became more humified and had a higher electricity generation compared to the sediment samples from the algae-dominated area. Pyrosequencing analysis further determined that SMFC promoted more aromatic compound-degrading bacteria growth in sediments from the macrophyte-dominated area than from the algae-dominated area. Our study demonstrated that SMFC could enhance organic matter degradation, especially plant litter degradation, but this influence showed different from sediment sources. Thus, SMFC is capable of providing a useful strategy for delaying the terrestrialization of lakes areas suffering from eutrophication.