The global decline of coral reefs is a major contributor to the global biodiversity crisis and requires improved monitoring at these critically important habitats. Non-invasive passive acoustic assessments may address this need, leveraging the rich variety and spatiotemporal variability of biological sounds present in coral reef environments and offering near-continuous temporal coverage. Despite this, acoustic metrics that reliably represent coral reef health are still debated, and ground-truthing of methods is limited. Here we investigated how the prevalence of low frequency biotic sounds (without species information) relates to coral reef health, providing a foundation from which one can compare assessment methods. We first quantified call rates of these low frequency sounds for three reefs exhibiting different community assemblages around St. John, U.S. Virgin Islands, by manually annotating presumed fish noises for 1 min every 30 min across 8 days for each site. Annotated days were selected at key points across lunar cycles. These call rates were then compared with traditional visual surveys, and several acoustic methods and indices commonly used in underwater soundscape research. We found that, overall, manually detected fish call rates successfully differentiated between the three reefs, capturing variation in crepuscular activity levels–a pattern consistent with previous work that highlights the importance of diel choruses. Moreover, fish vocal rates were predictors of hard coral cover, fish abundance, and fish species richness, while most acoustic indices failed to parse out fine distinctions among the three sites. Some, such as the Acoustic Complexity Index, failed to reveal any expected differences between sites or times of day, while the Bioacoustic Index could only identify the most acoustically active reef, otherwise having weak correlations to visual metrics. Of the indices tested, root-mean-squared sound pressure level and Acoustic Entropy, both calculated in the low frequency fish band (50–1,200 Hz), showed the strongest association with visual health measures. These findings present an important step toward using soundscape cues for reef health assessments. The limited generalizability of acoustic indices across different locations emphasizes the need for caution in their application. Therefore, it is crucial to improve methods utilizing fish sounds, such as automatic fish call detectors that are able to generalize well to new soundscapes.