All-Solid-State Lithium Batteries (ASSLBs) are promising since they may enable the use of high potential materials as positive electrode and lithium metal as negative electrode. This is only possible through solid electrolytes (SEs) stated large electrochemical stability window (ESW). Nevertheless, reported values for these ESWs are very divergent in the literature. Establishing a robust procedure to accurately determine SEs’ ESWs has therefore become crucial. Our work focuses on bringing together theoretical results and an original experimental set up to assess the electrochemical stability window of the two NASICON-type SEs Li1.3Al0.3Ti1.7(PO4)3 (LATP) and Li1.5Al0.5Ge1.5(PO4)3 (LAGP). Using first principles, we computed thermodynamic ESWs for LATP and LAGP and their decomposition products upon redox potentials. The experimental set-up consists of a sintered stack of a thin SE layer and a SE-Au composite electrode to allow a large contact surface between SE and conductive gold particles, which maximizes the redox currents. Using Potentiostatic Intermittent Titration Technique (PITT) measurements, we were able to accurately determine the ESW of LATP and LAGP solid electrolytes. They are found to be [2.65–4.6 V] and [1.85–4.9 V] for LATP and LAGP respectively. Finally, we attempted to characterize the decomposition products of both materials upon oxidation. The use of an O2 sensor coupled to the electrochemical setup enabled us to observe operando the production of O2 upon LAGP and LATP oxidations, in agreement with first-principles calculations. Transmission Electron Microscopy (TEM) allowed to observe the presence of an amorphous phase at the interface between the gold particles and LAGP after oxidation. Electrochemical Impedance Spectroscopy (EIS) measurements confirmed that the resulting phase increased the total resistance of LAGP. This work aims at providing a method for an accurate determination of ESWs, considered a key parameter to a successful material selection for ASSLBs.