We characterize the 3-D spatial variations of [Fe/H], [Mg/H], and [Mg/Fe] in stars at the time of their formation, across 11 simulated Milky Way (MW)-and M31-mass galaxies in the FIRE-2 simulations, to inform initial conditions for chemical tagging. The overall scatter in [Fe/H] within a galaxy decreased with time until ≈ 7 Gyr ago, after which it increased to today: this arises from a competition between a reduction of azimuthal scatter and a steepening of the radial gradient in abundance over time. The radial gradient is generally negative, and it steepened over time from an initially flat gradient 12 Gyr ago. The strength of the present-day abundance gradient does not correlate with when the disk 'settled'; instead, it best correlates with the radial velocity dispersion within the galaxy. The strength of azimuthal variation is nearly independent of radius, and the 360 degree scatter decreased over time, from 0.17 dex at t lb = 11.6 Gyr to ∼ 0.04 dex at present day. Consequently, stars at t lb 8 Gyr formed in a disk with primarily azimuthal scatter in abundances. All stars formed in a vertically homogeneous disk, ∆[Fe/H]≤ 0.02 dex within 1 kpc of the galactic midplane, with the exception of the young stars in the inner ≈ 4 kpc at z ∼ 0. These results generally agree with our previous analysis of gas-phase elemental abundances, which reinforces the importance of cosmological disk evolution and azimuthal scatter in the context of stellar chemical tagging. We provide analytic fits to our results for use in chemical-tagging analyses.