We study optimal bailout policies amidst banking and sovereign crises. Our model features sovereign borrowing with limited commitment, where domestic banks hold government debt and extend credit to the private sector. Bank capital shocks can trigger banking crises, prompting the government to consider extending guarantees over bank assets. This poses a trade-off: Larger bailouts relax financial frictions and increase output, but increase fiscal needs and default risk (creating a 'diabolic loop'). Optimal bailouts exhibit clear properties. The fraction of banking losses the bailouts cover is (i) decreasing in government debt; (ii) increasing in aggregate productivity; and (iii) increasing in the severity of banking crises. Even though bailouts mitigate the adverse effects of banking crises, the economy is ex ante better off without bailouts: Having access to bailouts lowers the cost of defaults, which in turn increases the default frequency, and reduces the levels of debt, output, and consumption.