Aims. The large and small-scale (pc) structure of the Galactic interstellar medium can be investigated by utilising spectra of earlytype stellar probes of known distances in the same region of the sky. This paper determines the variation in line strength of Ca ii at 3933.661 Å as a function of probe separation for a large sample of stars, including a number of sightlines in the Magellanic Clouds. Methods. FLAMES-GIRAFFE data taken with the Very Large Telescope towards early-type stars in 3 Galactic and 4 Magellanic open clusters in Ca ii are used to obtain the velocity, equivalent width, column density, and line width of interstellar Galactic calcium for a total of 657 stars, of which 443 are Magellanic Cloud sightlines. In each cluster there are between 43 and 111 stars observed.Additionally, FEROS and UVES Ca ii K and Na i D spectra of 21 Galactic and 154 Magellanic early-type stars are presented and combined with data from the literature to study the calcium column density -parallax relationship.Results. For the four Magellanic clusters studied with FLAMES, the strength of the Galactic interstellar Ca ii K equivalent width on transverse scales from ∼0.05−9 pc is found to vary by factors of ∼1.8−3.0, corresponding to column density variations of ∼0.3−0.5 dex in the optically-thin approximation. Using FLAMES, FEROS, and UVES archive spectra, the minimum and maximum reduced equivalent widths for Milky Way gas are found to lie in the range ∼35−125 mÅ and ∼30−160 mÅ for Ca ii K and Na i D, respectively.The range is consistent with a previously published simple model of the interstellar medium consisting of spherical cloudlets of filling factor ∼0.3, although other geometries are not ruled out. Finally, the derived functional form for parallax (π) and Ca ii column density (N CaII ) is found to be π(mas) = 1/(2.39 × 10 −13 × N CaII (cm −2 ) + 0.11). Our derived parallax is ∼25 per cent lower than predicted by Megier et al. (2009, A&A, 507, 833) at a distance of ∼100 pc and ∼15 percent lower at a distance of ∼200 pc, reflecting inhomogeneity in the Ca ii distribution in the different sightlines studied.