Acoustic energy is a type of environmental energy source that can be scavenged and converted into electrical energy for small-scale power applications. In general, incident sound power density is low and structural design for acoustic energy harvesting (AEH) is crucial. This review article summarizes the mechanisms of AEH, which include the Helmholtz resonator approach, the quarter-wavelength resonator approach, and the acoustic metamaterial approach. The details of recently proposed AEH devices and mechanisms are carefully reviewed and compared. Because acoustic metamaterials have the advantages of compactness, effectiveness, and flexibility, it is suggested that the emerging metamaterial-based AEH technique is highly suitable for further development. It is demonstrated that the AEH technique will become an essential part of the environmental energy-harvesting research field. As a multidisciplinary research topic, the major challenge is to integrate AEH devices into engineering structures and make composite structures smarter to achieve large-scale AEH.