For polarized remote sensors, the polarization images of ground objects acquired at different spatial scales will be different due to the spatial heterogeneity of the ground object targets and the limitation of imaging resolution. In this paper, the quantitative inversion problem of a typical polarized remote sensor at different spatial scales was studied. Firstly, the surface roughness of coatings was inversed based on the polarized bidirectional reflectance distribution function (pBRDF) model according to their polarization images at different distances. A linear-mixed pixel model was used to make a preliminary correction of the spatial scale effect. Secondly, the super-resolution image reconstruction of the polarization imager was realized based on the projection onto convex sets (POCS) method. Then, images with different resolutions at a fixed distance were obtained by utilizing this super-resolution image reconstruction method and the optimal spatial scale under the scene can be acquired by using information entropy as an evaluation indicator. Finally, the experimental results showed that the roughness inversion of coatings has the highest accuracy in the optimal spatial scale. It has been proved that our proposed method can provide a reliable way to reduce the spatial effect of the polarized remote sensor and to improve the inversion accuracy.