Jupiter's tropospheric composition and cloud structure are studied using Cassini VIMS 4.5-5.1 µm thermal emission spectra from the 2000-2001 flyby. We make use of both nadir and limb darkening observations on the planet's nightside, and compare these with dayside observations. Although there is significant spatial variability in the 5-μm brightness temperatures, the shape of the spectra remain very similar across the planet, suggesting the presence of a spectrally-flat, spatially inhomogeneous cloud deck. We find that a simple cloud model consisting of a single, compact cloud is able to reproduce both nightside and dayside spectra, subject to the following constraints: (i) the cloud base is located at pressures of 1.2 bar or lower; (ii) the cloud particles are highly scattering; (iii) the cloud is sufficiently spectrally flat. Using this cloud model, we search for global variability in the cloud opacity and the phosphine deep volume mixing ratio. We find that the vast majority of the 5-μm inhomogeneity can be accounted for by variations in the thickness of the cloud decks, with huge differences between the cloudy zones and the relatively cloud-free belts. The relatively low spectral resolution of VIMS limits reliable retrievals of gaseous species, but some evidence is found for an enhancement in the abundance of phosphine at high latitudes.