The suppression of low-frequency vibration and noise has always been an important issue in a wide range of engineering applications. To address this concern, a novel square hierarchical honeycomb metamaterial capable of reducing low-frequency noise has been developed. By combining Bloch’s theorem with the finite element method, the band structure is calculated. Numerical results indicate that this metamaterial can produce multiple low-frequency bandgaps within 500 Hz, with a bandgap ratio exceeding 50%. The first bandgap spans from 169.57 Hz to 216.42 Hz. To reveal the formation mechanism of the bandgap, a vibrational mode analysis is performed. Numerical analysis demonstrates that the bandgap is attributed to the suppression of elastic wave propagation by the vibrations of the structure’s two protruding corners and overall expansion vibrations. Additionally, detailed parametric analyses are conducted to investigate the effect of θ, i.e., the angle between the protruding corner of the structure and the horizontal direction, on the band structures and the total effective bandgap width. It is found that reducing θ is conducive to obtaining lower frequency bandgaps. The propagation characteristics of elastic waves in the structure are explored by the group velocity, phase velocity, and wave propagation direction. Finally, the transmission characteristics of a finite periodic structure are investigated experimentally. The results indicate significant acceleration amplitude attenuation within the bandgap range, confirming the structure’s excellent low-frequency vibration suppression capability.