Regardless of whether the global navigation satellite system (GNSS)/inertial navigation system (INS) is integrated or the INS operates independently during GNSS outages, the stochastic error of the inertial sensor has an important impact on the navigation performance. The structure of stochastic error in low-cost inertial sensors is quite complex; therefore, it is difficult to identify and separate errors in the spectral domain using classical stochastic error methods such as the Allan variance (AV) method and power spectral density (PSD) analysis. However, a recently proposed estimation, based on generalized wavelet moment estimation (GMWM), is applied to the stochastic error modeling of inertial sensors, giving significant advantages. Focusing on the online implementation of GMWM and its integration within a general navigation filter, this paper proposes an algorithm for online stochastic error calibration of inertial sensors in urban cities. We further develop the autonomous stochastic error model by constructing a complete stochastic error model and determining model ranking criterion. Then, a detecting module is designed to work together with the autonomous stochastic error model as feedback for the INS/GNSS integration. Finally, two experiments are conducted to compare the positioning performance of this algorithm with other classical methods. The results validate the capability of this algorithm to improve navigation accuracy and achieve the online realization of complex stochastic models.