The Extragalactic Background Light (EBL) that spans the UV-IR band originates from direct and dust-reprocessed starlight integrated over the history of the Universe. EBL measurements are very challenging due to foreground emission like the zodiacal light and interplanetary dust emission. Indeed, some optical/NIR direct measurements overpredict EBL models based on galaxy counts. On the other hand, there is some debate on possible additional components of the Optical-NIR photon density: e.g., population-III stars, axion-photon decay, direct collapse of black holes, intra-halo light etc. Owing to the absorption of Very High Energy (VHE) γ rays by interaction with EBL photons, we study the prospects of accommodating an additional population of EBL sources in the Optical-NIR band on top of the standard galaxy-count–based component. To this aim we use 105 VHE spectra of 37 blazars with known redshifts, 0.03 < z < 0.94. We correct the observed spectra for absorption by our model EBL. By requiring the intrinsic spectra to be non-concave and with a VHE spectral index >1.5, we estimate, at different wavelengths, upper limits to the additional low energy photon fields which would contribute to the absorption of γ-rays. Considering these limits we suggest that there is room for photons from Pop III stars and axion-like particle (ALP) annihilation. However, these additional hypothetical photon fields are bound to fall significantly below direct published EBL measurements by several instruments, and therefore our limits are either in tension or even inconsistent with such measurements.