Interlayer excitons in transition-metal dichalcogenide (TMD) bilayers, alongside their interplay with direct excitonic species, are supposed to offer a pathway towards robust nonlinearity, enabling the exploration of many-body quantum effects. We present a theoretical investigation of interaction among various exciton species within these structures where Coulomb attraction and repulsion are subject to reduced screening. For a homobilayer MoS2, we examine both direct, spatially-indirect, and hybridised excitons, considering the effects of direct and exchange interaction of electrons and holes distributed across one or different layers. Similar physics arises in perfectly aligned twisted TMD heterobilayers which support the direct-to-indirect exciton hybridisation. Deriving the exciton-exciton interaction matrix elements, we unveil a distinct non-monotonic dependence of the interaction on transferred momentum, changing sign from repulsive to attractive even for ground-state excitons, and compare our results with existing calculations for monolayers. Our findings demonstrate that for large momenta involved in high-density effects (strongly correlated phases), the interaction is chiefly governed by the prevailing attractive exchange component. At the same time, at small momenta that are more relevant for rarefied systems, we find that the enhancement of the interaction constant for dipolar species compared to intralayer non-dipolar excitons may be hindered by the surrounding medium. We draw comparisons with existing experiments and discuss the implications of our findings on the collective effects in TMD-based systems of excitons and exciton-polaritons.