The orbital angular momentum (OAM) conversion is critical in understanding the interaction between the structural sound field and a planar lattice. Herein, we explored the evolution of a monochromatic acoustic vortex beam (AVB) that is scattered by a phononic crystal (PnC) or a correlated random lattice. The phenomenon is ascribed to the enhanced orbit-orbit angular momentum coupling induced by the band structure. By modifying the coupling condition, accurate and continuous micro-manipulation of the AVBs can be achieved, including the transverse/lateral gravity shift, the dynamics of the phase singularities, the spatial distribution of acoustic pressure and etc. This research provides insight to the inhomogeneous coupling of AVBs with both the propagating Bloch waves and the localized Anderson modes, and may facilitate the development of novel OAM-based acoustic devices for active sound field manipulation.