The cumulative emission of Axion-Like Particles (ALPs) from all past core-collapse supernovae (SNe) would lead to a diffuse flux with energies O(50) MeV. We use this to constrain ALPs featuring couplings to photons and to nucleons. ALPs coupled only to photons are produced in the SN core via the Primakoff process, and then converted into gamma rays in the Galactic magnetic field. We set a bound on gaγ 5 × 10 −10 GeV −1 for ma 10 −11 eV, using recent measurements of the diffuse gamma-ray flux observed by the Fermi-LAT telescope. However, if ALPs couple also with nucleons, their production rate in SN can be considerably enhanced due to the ALPs nucleonnucleon bremsstrahlung process. Assuming the largest ALP-nucleon coupling phenomenologically allowed, bounds on the diffuse gamma-ray flux lead to a much stronger gaγ 6 × 10 −13 GeV −1 for the same mass range. If ALPs are heavier than ∼ keV, the decay into photons becomes significant, leading again to a diffuse gamma-ray flux. In the case of only photon coupling, we find, e.g. gaγ 5 × 10 −11 GeV −1 for ma ∼ 5 keV. Allowing for a (maximal) coupling to nucleons, the limit improves to the level of gaγ 10 −19 GeV −1 for ma ∼ 20 MeV, which represents the strongest constraint to date.