Due to the evolving nature of power grids and model uncertainty, the online stability assessment of electrical power systems is always a challenging problem. This paper aims to provide a theoretical framework for estimating the region of attraction for power systems in real time. By analyzing and learning the measurement data in a given time duration, a windowed online Gaussian process (GP) approach is developed to provide the real-time security assessment and quantify the uncertainty caused by measurement errors and dynamic evolution of power systems. In addition, the theoretical analysis is conducted to ensure the conference level of the estimated region of attraction. Finally, numerical simulations are implemented on a microgrid model with 9 buses and 3 generators to validate the proposed approach. The proposed online assessment approach contributes to improving the situational awareness of human operators, thereby taking remedial actions before the emergency.