Protective seam mining is the most economical and effective measure for eliminating coal and gas herniation. To study the unloading effect of the mining of a protective seam on the protected layer, and to better grasp the effect of the protective layer on the abatement, conventional triaxial tests were conducted on coal samples with the unloading of the axial pressure and the peripheral pressure. The results showed that, under the unloading path, the bias stress–axial strain curve showed a sudden upward trend upon unloading, and the slope of the curve increased suddenly, which was more obvious after the peripheral pressure exceeded 10 MPa; stress unloading before the peak accelerated the yielding of the specimen. Under the unloading test path, the deformation modulus of the coal samples decreased with the decrease in the perimeter pressure, while the damage factor and Poisson’s ratio increased with the decrease in the perimeter pressure. Compared to the conventional triaxial test, under the unloading condition, the cohesion of the coal samples at peak stress decreased by 93.41% and the angle of internal friction increased by 37.41%, while the cohesion at the moment of residual strength decreased by 89.60% and the angle of internal friction increased by 37.44°. The brittleness index of the coal samples under unloading conditions with a peripheral pressure of 5 MPa, 10 MPa, 15 MPa, and 20 MPa increased by 178.83%, 159.18%, 87.93%, and 63.89%, respectively, compared to the conventional triaxial test. It can be seen that the greater the enclosing pressure, the smaller the difference in the brittleness index of the coal body.