Moments of moments of the Riemann zeta function, defined by $$ \text{MoM}_T(k,\beta) := \frac{1}{T}\int_T^{2T} \Bigg(\,\int\limits_{ |h|\leq (\log T)^\theta}|\zeta(\frac{1}{2} + i t + ih)|^{2\beta}\ dh\Bigg)^k\ dt, $$ where $k,\beta \geq 0$ and $\theta \gt -1$ were introduced by Fyodorov and Keating, Freezing transitions and extreme values: random matrix theory, and disordered landscapes, Philos. Trans. Roy. Soc. A: 372 no. 2007 (2014), 20120503 A doi:10.1098/rsta.2012.0503 when comparing extreme values of zeta in short intervals to those of characteristic polynomials of random unitary matrices. We study the k = 2 case as $T \rightarrow \infty$ and obtain sharp upper bounds for $\text{MoM}_T(2,\beta)$ for all real $0\leq \beta \leq 1$ as well as lower bounds of the conjectured order for all $\beta \geq 0$. In particular, we show that the second moment of moments undergoes a freezing phase transition with critical exponent $\beta = \frac{1}{\sqrt{2}}$. The main technical ingredient is a new estimate for the correlation of two shifted $2\beta^{\text{th}}$ powers of zeta with $0\leq \beta \leq 1$.