Soundscape analysis is a potentially powerful tool in ecosystem monitoring. Ecoacoustic metrics, including the Acoustic Complexity Index (ACI) and Acoustic Entropy (H), were originally developed for terrestrial ecosystems and are now increasingly being applied to investigate the biodiversity, habitat complexity and health of marine systems, with mixed results. To elucidate the efficacy of applying these metrics to marine soundscapes, their sensitivity to variations in call rate and call type were evaluated using a combination of field data and synthetic recordings. In soundscapes dominated by impulsive broadband snapping shrimp sounds, ACI increased non-linearly with increased snapping rate (∼100–3500 snaps/min), with a percent range of variation (∼40–50%) that exceeds that reported in most studies. H, however, decreased only slightly (∼0.04 units) in response to these same snap rate changes. The response of these metrics to changes in the rate of broadband snapping was not strongly influenced by the spectral resolution of the analysis. For soundscapes dominated by harmonic fish calls, increased rates of calling (∼5–120 calls/min) led to decreased ACI (∼20–40% range of variation) when coarse spectral resolutions (Δf = 94 or 47 Hz) were used in the analysis, but ACI increased (∼20% range of variation) when a finer resolution (Δf = 23 Hz) was employed. Regardless of spectral resolution used in the analysis, H decreased (∼0.20 units) in response to increased rates of harmonic calling. These results show that ACI and H can be modulated strongly by variations in the activity of a single sound-producing species, with additional sensitivity to call type and the resolution of the analysis. Variations in ACI and H, therefore, cannot be assumed to track call diversity, and the utility of these metrics as ecological indicators in marine environments may be limited.