In recent years, detailed observations and accurate numerical simulations have provided support to the idea that mergers of compact binaries containing either two neutron stars (NSs) or an NS and a black hole (BH) may constitute the central engine of short gamma-ray bursts (SGRBs). The merger of such compact binaries is expected to lead to the production of a spinning BH surrounded by an accreting torus. Several mechanisms can extract energy from this system and power the SGRBs. Here we connect observations and numerical simulations of compact binary mergers, and use the current sample of SGRBs with measured energies to constrain the mass of their powering tori. By comparing the masses of the tori with the results of fully generalrelativistic simulations, we are able to infer the properties of the binary progenitors which yield SGRBs. By assuming a constant efficiency in converting torus mass into jet energy, ǫ jet = 10%, we find that most of the tori have masses smaller than 0.01M ⊙ , favoring "high-mass" binary NSs mergers, i.e., binaries with total masses 1.5 the maximum mass of an isolated NS. This has important consequences for the gravitational-wave signals that may be detected in association with SGRBs, since "high-mass" systems do not form a long-lived hypermassive NS after the merger. While NS-BH systems cannot be excluded to be the engine of at least some of the SGRBs, the BH would need to have an initial spin of ∼ 0.9, or higher.