The shale of the Wufeng–Longmaxi formation in the Sichuan Basin is the preferred layer for shale gas exploration in China, and its petrophysical characteristics are the key to geological and engineering sweet spot prediction. However, the characteristics and impact mechanisms of its acoustic wave velocity and elastic anisotropy are currently unclear. In this paper, the Wufeng–Longmaxi shale is taken as the research object, and the P-wave and S-wave velocities of the samples are tested under the loading and unloading processes of confining pressure. The stress sensitivity variations in parameters such as wave velocity, wave velocity ratio, and anisotropy are discussed. P-wave and S-wave anisotropy parameters are correlated under different pressure conditions. X-ray diffraction, casting thin sections, scanning electron microscopy, micron CT scanning, and other analytical techniques are used to explore the mechanisms of stress sensitivity of elastic parameters. The research results indicate that: (1) the acoustic velocities of samples from different angles are V90° > V45° > V0°, and there is a positive correlation between the wave velocity and the confining pressure. After unloading the confining pressure, irreversible plastic deformation occurs due to the closure of some microfractures in the rock core, causing the wave velocity to be higher than the initial value. (2) The stress sensitivity coefficient of the P-wave (The mean is 3.00 m·s−1·MPa−1) is higher than that of the S-wave (the mean is 1.23 m·s−1·MPa−1), and the stress sensitivity coefficient of the compacted stage (the mean is 3.02 m·s−1·MPa−1) is higher than that of the elastic stage (the mean is 1.21 m·s−1·MPa−1). (3) The anisotropy of the P-wave and S-wave is negatively correlated with the confining pressure. When the confining pressure is loaded to 65 MPa, the change rate of the P-wave anisotropy coefficient is 23%, and its stress sensitivity is higher than that of S-wave anisotropy coefficient (the change rate is 13.7%). After unloading the confining pressure, the degree of anisotropy is reduced due to the closure of some microfractures. The empirical formula of P-wave and S-wave anisotropy parameters under different pressures is established through linear regression, which can provide a reference for mutual predictions. (4) The variation in wave velocity anisotropy with stress can be divided into stress and material anisotropy, which are related to the directional arrangement of microfractures and clay minerals, respectively. The quantitative characterization of shale anisotropy can be realized by evaluating the development degree of reservoir fractures and mineral components, providing a reference for logging interpretations, sweet spot prediction, and fracturing construction of shale gas reservoirs.