We have investigated here the structure of the pelagic microbial food web and quantified the carbon fluxes from viruses to microplankton along trophic gradients in the Mediterranean Sea. To explore the complex trophic pathways of the pelagic food web, we conducted independent and replicated experiments to measure (i) predation on prokaryotes by microzooplankton, (ii) predation on prokaryotes by heterotrophic nanoflagellates, (iii) virus‐induced prokaryotic mortality, and (iv) microzooplankton grazing on nanoplankton and microphytoplankton. Our study covered more than 5000 km, from the Atlantic Ocean to the Levantine basin, and from conditions of high primary production and nutrient availability to ultraoligotrophic and phosphate‐limited waters. Microphytoplankton abundance and biomass were typically scarce across the entire Mediterranean basin, with almost negligible levels in the eastern part. Also, nanoplankton biomass was typically low. Conversely, prokaryotes, and particularly the heterotrophic components, were abundant and represented the only significant food source for both nanoplankton and microplankton grazers. Viral infections were not the primary agents of prokaryotic mortality, but in some areas, such as the Ligurian Sea, they had a key role in prokaryotic dynamics. The scenario depicted in this study in summer reveals the pivotal role of microzooplankton in the pelagic food web of the Mediterranean Sea, with a key role in the potential transfer of biomass to higher trophic levels. We also show that converse to theoretical expectations, the microbial food web was relatively complex under the mesotrophic conditions (Atlantic and western Mediterranean) and was much more simplified in the ultraoligotrophic conditions of the eastern Mediterranean.