Research on the acoustic stealth structures of submarines primarily strives for low frequency, pressure resistance, and broad frequency domain absorption. The high-quality acoustic stealth coating on submarines is a complicated Multi-scale and multi-component structure, which consists of numerous macro-structures, such as acoustic air-filled cavities, laminated structures, filling components, and micro-phase structures. However, more information regarding how these components with different anechoic mechanisms obtain good results through synergistic work and how their absorption frequencies superpose effectively to realize acoustic stealth in a broad frequency domain is necessary. Understanding these mechanisms is essential to design a novel anechoic structure in a broad frequency domain. Thus, in this paper, research concerning these mechanisms was reviewed. In addition, the design methods and anechoic mechanism of anechoic structures were discussed for different levels, and the primary concerns in this field were summarized. Several suggestions were made for further research.