Given a finite-to-one map T acting on a compact metric space Ω and an appropriate Banach space of functions X (Ω), one classically constructs for each potential A ∈ X a transfer operator L A acting on X (Ω). Under suitable hypotheses, it is well-known that L A has a maximal eigenvalue λ A , has a spectral gap and defines a unique Gibbs measure µ A . Moreover there is a unique normalized potential of the form B := A + f − f • T + c acting as a representative of the class of all potentials defining the same Gibbs measure.The goal of the present article is to study the geometry of the set of normalized potentials N , of the normalization map A → B, and of the Gibbs map A → µ A . We give an easy proof of the fact that N is an analytic submanifold of X and that the normalization map is analytic; we compute the derivative of the Gibbs map; last we endow N with a natural weak Riemannian metric (derived from the asymptotic variance) with respect to which we compute the gradient flow induced by the pressure with respect to a given potential, e.g. the metric entropy functional. We also apply these ideas to recover in a wide setting existence and uniqueness of equilibrium states, possibly under constraints.