For the acousto-optic tunable filter (AOTF)-based spectral imaging systems, the diffraction efficiency of the AOTF is a primary factor affecting system throughput. Moreover, the distribution of the acoustic field within the AOTF fundamentally determines the device’s diffraction efficiency. Thus, the design of an AOTF device including a transducer and absorber to achieve a uniform acoustic field distribution plays an important role in improving diffraction efficiency. This study proposed an acoustic absorbing structure using mercurous halide crystals’ strong acoustic anisotropy to realize the conversion from shear horizontal wave to surface wave at the boundary and rapid dissipation. Snell’s law for acoustically anisotropic media is employed to design the angle of the acoustic absorbing structure. Experiments of mercurous halide-based AOTF demonstrate that this absorbing structure can effectively enhance diffraction efficiency.