Sapphire is an important material for optical applications, but the anisotropy of key properties makes it difficult to obtain uniform ground surface. In this work, cross-grinding experiments were first conducted on C- and M-planes of sapphire to investigate the dependence of the ground surface quality on crystal orientation. Then, scratching tests were performed along the specific crystal orientations to explain the formation mechanism of the anisotropic grinding patterns. The results showed that the surface roughness distribution on C-plane is of a threefold symmetry, while one axis of symmetry for M-plane. The damage pits on C-plane possess lower aspect ratio than that on M-plane. By combining the scratching-induced fracture features with the contact-induced deformation theory of sapphire, it was found that the propagation of the primary cracks in sapphire is accompanied by activating different deformation systems. The material removal on C-plane is dominated by the shallow lateral cracks, which propagate parallel to the ground surface and result in shallow and wide pits. However, the deep lateral cracks play an important role on M-plane, which intersect with the ground surface and cause deep and dense defects. The non-uniformity of the ground surface is attributed to the appearance of radial cracks for C-plane, while it is associated with the number of the available deformation systems for M-plane.