In this paper, we conduct a comprehensive investigation into PVA fiber modified with SiO2 to improve the mechanical properties of oil-well cements. Specifically, SiO2 was coated onto the surface of polyvinyl alcohol fiber (PVAF) as its silicon source via a sol-gel process by using tetraethyl orthosilicate (TEOS), while hydrochloric acid and ammonia were respectively used as the catalyst in the sol (hydrolysis) and the gel (condensation) processes. The PVAF microstructure was then characterized with the scanning electron microscope (SEM), while the effects of the modified PVAF on both mechanical and rheological properties of oil-well cements were examined. Due to the fact that SiO2 can be uniformly coated onto the PVAF surface, such modified PVAF can slightly improve the rheology of the cement slurry, while the raw PVAF exhibits poor dispersion at a high dosage. Compared with those of cement stone without PVAF after curing for 28 days at 60 °C, the flexural strength, compressive strength, and elastic modulus of the cement stone incorporated with the modified PVAFs were enhanced by 37.7%, 66.1%, and 50.0%, respectively. The SEM test (EDX) test, XRD test, and thermogravimetric test prove that the SiO2 coating on the PVAF surface can promote the hydration of cement clinker and can react with Ca(OH)2 to generate CSH gel. The SiO2 grafted onto the surface of PVAFs can improve the bond strength at the fiber/cement matrix interface, thus improving the mechanical properties of cement stone.