The statistical characterization of the cavity quality factor (Q-factor) holds significant practical importance, especially in the context of large and complex metallic enclosures. While there are existing methods for analyzing Q-factor statistics attributed to distributed and uniform losses, there is a noticeable gap in addressing the statistical cavity Q-factor arising from localized losses, such as aperture leakage, wall coating, and absorptive loading. This article introduces a physics-oriented, hybrid deterministic-stochastic approach to predict the statistical distribution of cavity Q-factor due to localized losses. The key ingredient of this method is the stochastic Green's function integral equation formulation, grounded in a statistical description of the cavity eigenmodes within an enclosed electromagnetic environment. The computational model is evaluated through both numerical and laboratory experiments, validating its reliability and applicability in real-world scenarios.