We present a new characterisation of Goursat categories in terms of special kind of pushouts, that we call Goursat pushouts. This allows one to prove that, for a regular category, the Goursat property is actually equivalent to the validity of the denormalised 3-by-3 Lemma. Goursat pushouts are also useful to clarify, from a categorical perspective, the existence of the quaternary operations characterising 3-permutable varieties.