Using photometric variability information from the new Gaia DR3 release, I show for the first time that photometric variability is inversely correlated with the prevalence of optical–radio position offsets in the active galactic nuclei (AGNs) that comprise the International Celestial Reference Frame (ICRF). While the overall prevalence of statistically significant optical–radio position offsets is 11%, objects with the largest fractional variabilities exhibit an offset prevalence of only ∼2%. These highly variable objects have redder optical color and steeper optical spectral indices indicative of blazars, in which the optical and radio emission is dominated by a line-of-sight jet, and indeed nearly ∼100% of the most variable objects have γ-ray emission detected by the Fermi Large Area Telescope. This result is consistent with selection on variability preferentially picking jets pointed closest to the line of sight, where the projected optical–radio position offsets are minimized and jet emission is maximally boosted in the observed frame. While only ∼9% of ICRF objects exhibit such large photometric variability, these results suggest that taking source variability into account may provide a means of optimally weighting the optical–radio celestial reference frame link.