The rate of merger events observed by LIGO/Virgo can be used in order to probe the fraction f of dark mater in the form of primordial black holes (PBH). Here, we consider the merger rate of PBH binaries, accounting for the effect of cosmological perturbations on their initial eccentricity e. The torque on the binaries may receive significant contributions from a wide range of scales, that goes from the size of the horizon at the time when the binary forms, down to the co-moving size of the binary. Extrapolating the observed plateau in the power spectrum P Φ ≈ 10 −9 from cosmological scales down to the co-moving size of binaries, the torque from perturbations is small. In this case, for f 10 −2 , the distribution of eccentricities is dominated by tidal torques from neighboring PBHs. On the other hand, in scenarios where PBH are formed from adiabatic perturbations, it is natural to expect an enhancement of P Φ at small scales, where it is poorly constrained observationally. The effect can then be quite significant. For instance, a nearly flat spectrum with amplitude P Φ 10 −7 on scales smaller than ∼ 10M pc −1 gives a contribution j 2 ∼ 10 3 P Φ , where j = (1 − e 2 ) 1/2 is the dimensionless angular momentum parameter of the binaries. This contribution can dominate over tidal torques from neighboring PBHs for any value of f . Current constraints allow for a power spectrum as large as P Φ ∼ 10 −5 at the intermediate scales 10 3 − 10 5 M pc −1 , comparable to the co-moving size of the binaries at the time of formation. In particular, this can relax current bounds on the PBH abundance based on the observed LIGO/Virgo merger rate, allowing for a fraction f ∼ 10% of dark matter in PBH of mass ∼ 30M . We investigate the differential merger rate ∆Γ(m 1 , m 2 ), as a function of the masses of the binary components, and the corresponding "universality" coefficient [1] α = −(m 1 + m 2 ) 2 ∂ 2 ln ∆Γ/∂m 1 ∂m 2 . For an enhanced power spectrum with spectral index p we find that α ≈ 30/(32 − 7p) for 0 < p 2, and α ≈ 5/3 for p 2. Such values may lie well outside the narrow range α ≈ 1 ± 0.05 characteristic of tidal forces from neighboring PBHs. We conclude that, given a large enough sample of events, merger rates may provide valuable information on the spectrum of primordial cosmological perturbations at currently uncharted lengthscales.