Against the backdrop of concerns for diminishing the vulnerabilities of the economies of the Member States, the EU has adopted measures to strengthen budgetary discipline and control of the public deficit. In this context, the responsibility of government institutions has increased, not only in ensuring the sustainability of public finances but also in direct or indirect cooperation for good economic governance. From this perspective, this study aims to assess the impact of macroeconomic variables and those associated with supreme audit institutions on the sustainability of public finances measured by the size and dynamics of government deficit and gross public debt. Additionally, the impact of the same variables on governmental effectiveness and control of corruption has also been assessed. The data collected from secondary sources and panel data models were used to conduct an empirical study of the EU Member States which covered the 2002–2019 period and the sub-periods, divided as follows: pre-crisis, crisis, and post-crisis. The results of the study show that supreme audit institutions, through their organizational structure, the nature of their activities, and professionalism, may contribute to the reduction of public deficit and gross public debt and, implicitly, to higher efficiency and control of corruption. The results of analyses for the sub-periods show that ISAs played a more important role in reducing government deficit during crisis and post-crisis periods. By confirming or rejecting the results of the few studies that have been conducted so far, this study provides additional evidence that fills the gaps in the literature.