Marine food webs are structured through a combination of top‐down and bottom‐up processes. In coral reef ecosystems, fish size is related to life‐history characteristics and size‐based indicators can represent the distribution and flow of energy through the food web. Thus, size spectra can be a useful tool for investigating the impacts of both fishing and habitat condition on the health and productivity of coral reef fisheries. In addition, coral reef fisheries are often data‐limited and size spectra analysis can be a relatively cost‐effective and simple method for assessing fish populations. Abundance size spectra are widely used and quantify the relationship between organism size and relative abundance. Previous studies that have investigated the impacts of fishing and habitat condition together on the size distribution of coral reef fishes, however, have aggregated all fishes regardless of taxonomic identity. This leads to a poor understanding of how fishes with different feeding strategies, body size‐abundance relationships, or catchability might be influenced by top‐down and bottom‐up drivers. To address this gap, we quantified size spectra slopes of carnivorous and herbivorous coral reef fishes across three regions of Indonesia representing a gradient in fishing pressure and habitat conditions. We show that fishing pressure was the dominant driver of size spectra slopes such that they became steeper as fishing pressure increased, which was due to the removal of large‐bodied fishes. When considering fish functional groups separately, however, carnivore size spectra slopes were more heavily impacted by fishing than herbivores. Also, structural complexity, which can mediate predator‐prey interactions and provisioning of resources, was a relatively important driver of herbivore size spectra slopes such that slopes were shallower in more complex habitats. Our results show that size spectra slopes can be used as indicators of fishing pressure on coral reef fishes, but aggregating fish regardless of trophic identity or functional role overlooks differential impacts of fishing pressure and habitat condition on carnivore and herbivore size distributions.