Very high energy (VHE, energy E < ∼ 100 GeV) γ-rays from cosmological sources are attenuated due to the interaction with photons of the extragalactic background light (EBL) in the ultraviolet to infrared wavelength band. The EBL, thus, leaves an imprint on the observed energy spectra of these objects. In the last four years, the number of extragalactic VHE sources discovered with imaging atmospheric Cherenkov telescopes (IACTs), such as MAGIC, H.E.S.S., and VERITAS, has doubled. Furthermore, the measurements of the Fermi satellite brought new insights into the intrinsic spectra of the sources at GeV energies. In this paper, upper limits on the EBL intensity are derived by considering the most extensive VHE source sample ever used in this context. This is accomplished by constructing a large number of generic EBL shapes and combining spectral informations from Fermi and IACTs together with minimal assumptions about the source physics at high and very high γ-ray energies. The evolution of the EBL with redshift is accounted for and the possibility of the formation of an electromagnetic cascade and the implications on the upper limits are explored. The EBL density at z = 0 is constrained over a broad wavelength range between 0.4 and 100 μm. At optical wavelengths, the EBL density is constrained below 24 nW m −2 sr −1 and below 5 nW m −2 sr −1 between 8 μm and 31 μm.