In this study, a series of tribological tests were conducted on a pin-on-disc tester to study the lubrication mechanism of SiO2 nanoparticles under different surface roughness considering various loads and velocities. For a comprehensive understanding of the mechanism of SiO2 nanoparticles, base fluid was also employed as a contrast. Results show that the reductions of friction coefficients and wear scar widths increase with the decrease of surface roughness, due to the increase in rolling effect and self-repairing mechanism of SiO2 nanoparticles. The lubrication mechanism of SiO2 nanoparticles is the rolling effect when the height-diameter ratio (λ) is less than 6, and the self-repairing mechanism at λ of 6 and 10, whereas, there is no obvious difference by adding nanoparticles when λ is 20. When the height-diameter ratio is less than 6, surface wears show an increasing trend as the load increases due to the high hardness of nanoparticles, while it is the opposite at λ of 10 and 20 because of the self-repairing mechanism.