We study the secular coefficients of N × N random unitary matrices U N drawn from the Circular β-Ensemble, which are defined as the coefficients of {z n } in the characteristic polynomial det(1 − zU * N ). When β > 4 we obtain a new class of limiting distributions that arise when both n and N tend to infinity simultaneously. We solve an open problem of Diaconis and Gamburd [DG06] by showing that for β = 2, the middle coefficient of degree n = ⌊ N 2 ⌋ tends to zero as N → ∞. We show how the theory of Gaussian multiplicative chaos (GMC) plays a prominent role in these problems and in the explicit description of the obtained limiting distributions. We extend the remarkable magic square formula of [DG06] for the moments of secular coefficients to all β > 0 and analyse the asymptotic behaviour of the moments. We obtain estimates on the order of magnitude of the secular coefficients for all β > 0, and these estimates are sharp when β ≥ 2. These insights motivated us to introduce a new stochastic object associated with the secular coefficients, which we call Holomorphic Multiplicative Chaos (HMC). Viewing the HMC as a random distribution, we prove a sharp result about its regularity in an appropriate Sobolev space. Our proofs expose and exploit several novel connections with other areas, including random permutations, Tauberian theorems and combinatorics.