Depolarization at a rough surface relates to its roughness and irregularity (e.g., sags and crests) besides the material property. However, there is still lack of general theory to clearly describe the relationship between depolarization ratios and surface conditions, and one important reason is that the mechanism of depolarization relates to geometric parameters such as microcosmic height/particle distributions of sub-micro to nm levels. To study the mechanism in more detail, a compact laser instrument is developed, and depolarization information of a linearly polarized incident light is used for analyzing the roughness, during which a He-Ne laser source (λ = 632.8 nm) is used. Three nickel specimens with RMS roughness (Rq) less than λ/4 are fabricated and tested. Six different areas in each specimen are characterized in detail using an AFM. Rq are in the range of 34.1-155.0 nm, and the heights are non-Gaussian distribution in the first specimen and near-Gaussian distribution in the others. Off-specular inspection is carried out exactly on these 18 characterized areas, and results show that the cross-polarization ratios match quite well with Rq values of the first sample that has Rq ≤ λ/10 (or Rt ≤ λ), while they match well with maximum height, Rt, values of the other two that have Rt > λ (the maximum derivation is 11%). In addition, since this instrument is simple, portable, stable, and low-cost, it has great potential for practical online roughness testing after a linear calibration.