We study azimuthal asymmetries in diffractive J/ψ production in ultraperipheral heavy-ion collisions at RHIC and LHC energies using the color glass condensate effective theory. Our calculation successfully describes azimuthal averaged J/ψ production cross section measured by STAR and ALICE. We further predict very large cos 2φ and cos 4φ azimuthal asymmetries for diffractive J/ψ production both in UPCs at RHIC and LHC energies and in eA collisions at EIC energy. These novel polarization dependent observables may provide complementary information for constraining gluon transverse spatial distribution inside large nuclei. As compared to all previous analysis of diffractive J/ψ production, the essential new elements integrated in our theoretical calculations are: the double-slit interference effect, the linear polarization of coherent photons, and the final state soft photon radiation effect.