In this study, the effect of cyclic compression on the micromechanical properties of a Zr-based metallic glass (MG) was investigated via nanoindentation. Cyclic compression significantly softened the surface of the sample, with a maximum hardness loss of 19.93%. The number of cyclic compression passes had a greater effect on the hardness of the sample than the cyclic compression load. The elastic modulus exhibited a nonlinear variation upon increasing the cyclic loading or number of passes at a lower loading rate due to the coupling effect of loading rate and cyclic compression treatment. Then, the serration behavior and strain rate sensitivity analysis were applied. The calculated m-values obtained for MGs were all negative and gradually tended to zero upon further cyclic compression treatment. This demonstrated the weakening effect of cyclic compression on the strain rate sensitivity of MG, and the underlying mechanism was discussed. This study provides a process reference for studying the fatigue failure behaviors of MGs from the perspective of mechanical properties, which is useful for understanding their fatigue generation.