We carried out a joint theoretical and experimental study of the polarization of high-order harmonics generated from ZnO by intense infrared laser pulses. Experimentally we found that the dependence of parallel and perpendicular polarizations on the crystal orientation for all odd harmonics are nearly identical, but they are quite different from even harmonics which also show little order dependence. A one-dimensional two-band model, combined with a linear coupled excitation model, is shown to be able to explain the observed polarization behavior, including low-order harmonics. We further note that the same odd/even order contrast have been reported in a number of other crystals, despite that the harmonics were perceived to be generated via entirely different mechanisms. We demonstrated that this universality is governed by crystal symmetry, not by specific mechanisms. Thus, polarization measurements of harmonics offers a powerful pure optical method for determining the crystal axes as well as monitoring their ultrafast changes when crystals are undergoing deformation. In addition, the ellipticity of harmonic has been studied. It shows that ellipticity of high-order harmonics from solids can be tuned precisely by changing the bond structure of the sample.