Employing benchmarking metrics to capture the activity and stability of electrocatalysts for the oxygen evolution reaction (OER) in acid is a critical practice that enables meaningful comparison of catalyst material candidates reported throughout the literature. In this work, we find that ubiquitously used glassy carbon electrode substrates oxidize under typical OER operating conditions, forming a pacified, electrically insulating, and oxygen-rich surface layer that causes drastic loss of current density over the course of extended chronoamperometric stability tests at an anodic potential of 1.7 VRHE. We show that the experimentally observed stability of glassy carbon-based electrodes is approximately two orders of magnitude lower than that expected solely from dissolution-based catalyst intrinsic stability of Ir-based catalysts. We additionally find that glassy carbon-based electrode stability measured by chronoamperometric holds is greatly impacted by catalyst loading, with high catalyst loadings improving the stability of the overall electrode via a protective effect on the glassy carbon substrate. Overall, our investigation highlights that glassy carbon is not electrochemically inert under OER conditions on the timescale of common stability tests, which can cause electrodes to exhibit performance losses that do not reflect the intrinsic stability of the actual catalyst material being investigated. In light of our findings, we underscore the usefulness of metrics, such as the S-number, to reflect intrinsic catalyst material stability.