Fish often aggregate to spawn, feed, rest, or avoid predation. Direct observations of very high counts of large‐bodied grouper on deep shipwrecks, however, do not fit into typical descriptions of spawning‐, resource‐, or predation‐driven aggregations. To investigate whether these observations are rare or part of an underlying pattern, we synthesized four decades (1979–2019) of direct observations of groupers on deep‐water (50–300 m) habitats along the southeastern United States (Cape Hatteras, NC to Cape Canaveral, FL). The direct observations, which included 439 remotely operated vehicle transects, 235 human‐occupied vehicle transects, and 881 hook‐and‐line drops, revealed six hotspots of deep‐water groupers on three shipwrecks, two artificial reefs, and one boulder field. Grouper counts at these hotspots (0.10–5.40 grouper per linear m surveyed) exceeded counts of grouper outside of hotspots (<0.01–0.02 grouper per linear m surveyed) by multiple orders of magnitude. Commonalities among the sites with grouper hotspots included that all are relatively isolated structures surrounded by unconsolidated sediments and located in shelf‐edge to upper‐slope depths. Thus, it appears that these isolated habitats, despite their small spatial footprint, represent a disproportionate abundance of deep‐water groupers. Future research efforts should determine how groupers derive sufficient resources from, and thus co‐occur on, these small habitats and how these aggregations relate to the large‐scale dynamics of these populations.