The attitude algorithm is the most important part of the whole strapdown inertial navigation processing. It calculates the attitude of certain parameterization by integrating the gyro outputs or measurements in a specifically tailored way according to the attitude kinematic differential equation. The measurements or some angular velocity models obtained by fitting these measurements are often assumed free of errors in order to assess the numerical errors only. However, the gyro outputs and hence the models from them are by no means free of measurement errors. It is more often than not that the measurement errors dominate the numerical ones in practice. In this study, with coping with the measurement errors as the focus, we aim to improve the angular velocity model which is used as input in an attitude integration algorithm. This is achieved by exploiting the potential of overdetermined least-squares polynomial fitting. In order to avoid reducing the update rate by incorporating more measurements, the moving window trick is employed to re-use measurements in the previous update interval. The conventional attitude algorithm with second-order approximation in solving the differential equation of the equivalent rotation vector is employed as an example; however, the proposed method can be readily applied to other parameterizations such as direction cosine matrix, quaternion or Rodrigues parameters, and other high order approximations in solving the differential equation widely studied recently.