Quantum dots (QDs) embedded in inorganic matrices have been extensively studied for their potential applications in lighting, displays, and solar cells. While a significant amount of research focused on its experimental fabrication, the origin of their relatively low photoluminescence quantum yield has not been investigated yet, although it severely hinders practical applications. In this work, we use time-dependent density functional theory (TDDFT) to pinpoint the nature of excited states of CdSe quantum dots embedded in various inorganic matrices. The formation of undercoordinated Se atoms and non-bridging oxygen atoms at QD/glass interface is responsible for the localization of a hole wavefunction, leading to the formation of low- energy excited states with weak oscillator strength. These states provide pathways for non-radiative processes and compete with radiative emission. The photoluminescence performance is predicted for CdSe quantum dots in different matrices, and validated by experiments. The results of this work have significant implications for understanding the underlying photophysics of CdSe quantum dots embedded in inorganic matrices that would facilitate the fabrication of highly luminescent glasses.