Defenses that target particular consumers often influence community organization, ecosystem function, and diversity maintenance. In coral reef, mangrove, and seagrass ecosystems, sponges affect substratum stability, water clarity, diversity of associated species, and survival of habitat-providing organisms, key roles not duplicated by other organisms. Whether and how predators control sponges are much disputed. Substantial ecosystem consequences of losing or gaining sponges motivated definitive experiments on how predators control sponge distribution and abundance. Caribbean sponges of 94 species representing 13 taxonomic orders and three linked habitats (coral reefs, mangroves, and seagrass meadows) were exposed to seven predator species representing different habitats and degrees of spongivory in 4,493 in situ trials. The resulting data force reassessment of popular interpretations of several patterns and processes. Contrary to extract pellet assays that declare most sponges deterrent, 78% of these 94 species were eaten by at least one predator. But "palatability" is consumer dependent: a sponge species eaten by one predator can be rejected by other predators, and predator species differed in what sponges they ate in 55.4% (214/392) of pairwise comparisons between predators. Because spongivore species are usually restricted to particular habitats, they impose abrupt boundaries on sponges' habitat distributions, reflecting inverse relationships between accessibility and palatability to each predator. Thus a seagrass-dwelling starfish eats only 9% of seagrass sponge species, but 70% of coral reef species, and 78% of mangrove species. Reef-dwelling angelfishes completely consume only 13% of reef species, but 29% of seagrass species, and 63% of mangrove species. Defenses that target specific predators reveal that spongivore influence on community organization cannot be inferred from extract pellet/ omnivore assays that assume defenses target all predators equally. In fact, pellet data wrongly predicted actual consumption of living sponges of that pellet's species in 43% of field experiments with spongivores. In contrast with herbivore-plant interactions, opportunistic spongivory is at least as important as routine spongivory for community organization and ecosystem function. Potential for loss of key functional roles of sponges, if opportunistic predators gain access to sponge species that lack defenses against them, must inform conservation plans for coral reef, mangrove, and seagrass ecosystems.