In the search for topological phases in correlated electron systems, materials with 5d transitionmetal ions, in particular, the iridium-based pyrochlores A2Ir2O7, provide fertile grounds. Several novel topological states have been predicted but the actual realization of such states is believed to critically depend on the strength of local potentials arising from distortions of the IrO6 cages. We test this hypothesis by measuring with resonant inelastic x-ray scattering the electronic level splittings in the A=Y, Eu systems, which we show to agree very well with ab initio quantum chemistry electronic-structure calculations for the series of materials with A=Sm, Eu, Lu, and Y. We find, however, that not distortions of the IrO6 octahedra are the primary source for quenching the spin-orbit interaction, but longer-range lattice anisotropies which inevitably break the local cubic symmetry.