An important task for multi-agent systems (MAS) is to reach a consensus, e.g. to align their velocity vectors. Recent results propose appropriate consensus protocols to achieve such tasks, but most of them do not consider the effect of communication constraints such as the presence of time-delays in the exchange of information between the agents. In this article, we provide conditions for a non-linear, locally passive MAS of arbitrary size to reach a consensus, when the agents communicate over a packet-switched network that is characterised by a given topology. Both the cases of constant and switching topologies are considered. The nature of the communication channel imposes constraints that are modelled using stochastic delays of arbitrary distribution. We first embed this model in another, distributed but deterministic delay model and provide conditions for the error introduced by this simplification. In our main result, we provide conditions for the locally passive MAS with distributed delays to reach a consensus. In the case of a fixed topology, the underlying directed graph has to contain a spanning tree. In the case of a switching topology, only the union graph of all graphs that persist over time is required to contain a spanning tree. These conditions are independent of the distribution and the size of the packet delays. To show attractivity of the consensus set, we use an invariance principle for systems described by functional differential equations based on an appropriate Lyapunov-Razumikhin function. This methodological approach is the main contribution of this work and can also be applied to other consensus problems with delays. We illustrate our results by numerical simulations showing synchronisation of non-linear Kuramoto oscillators over a digital network.