Predator-prey dynamics can affect assemblage structure and ecosystem processes representing a central theory in ecology. In coral reef ecosystems, recent evidences have suggested that sponge assemblages in regions with high diversity, like the Caribbean, are controlled by reef fishes (i.e., top-down control); however, this has been poorly studied in low diversity coral reefs. This study investigated the influence of fish predators on sponge assemblage structure in South Atlantic coral reefs, systems with high endemism and relatively low hermatypic coral diversity. We investigates (i) whether sponge cover is negatively correlated to spongivorous fish density, (ii) potential spongivory effects on competitive interactions between sponges and hermatypic corals, and (iii) foraging preferences of spongivorous fishes. Benthic cover and spongivorous fish density were assessed by photo sampling and visual census, respectively. We did not observe a negative correlation of the total density of spongivorous fish with total sponge cover. However, a significant negative correlation between density of fish species Pomacanthus arcuatus and cover of sponge species Scopalina ruetzleri was found. Spongivorous fish consumed preferentially the sponges Desmapsamma anchorata, Niphates erecta, Aplysina cauliformis, and S. ruetzleri, the first two species considered palatable and the second two with chemically defense mechanism. An increase to angelfish density was not related with the number of coral-sponge encounters. Thus, the effects of spongivorous fishes on sponges cover and competitive interactions with hermatypic corals is weaker in Southwestern Atlantic than previously reported in Caribbean coral reefs. We discuss how local human impacts (e.g., fishing and nutrients input) can influence the observed patterns.