The ultrafast dynamics of hot carriers in graphene are key to both understanding of fundamental carrier-carrier interactions and carrier-phonon relaxation processes in two-dimensional materials, and understanding of the physics underlying novel high-speed electronic and optoelectronic devices. Many recent experiments on hot carriers using terahertz spectroscopy and related techniques have interpreted the variety of observed signals within phenomenological frameworks, and sometimes invoke extrinsic effects such as disorder. Here, we present an integrated experimental and theoretical programme, using ultrafast timeresolved terahertz spectroscopy combined with microscopic modelling, to systematically investigate the hot-carrier dynamics in a wide array of graphene samples having varying amounts of disorder and with either high or low doping levels. The theory reproduces the observed dynamics quantitatively without the need to invoke any fitting parameters, phenomenological models or extrinsic effects such as disorder. We demonstrate that the dynamics are dominated by the combined effect of efficient carrier-carrier scattering, which maintains a thermalized carrier distribution, and carrier-optical-phonon scattering, which removes energy from the carrier liquid.