The gravitational pull of an unseen companion to a luminous star is well known to cause deviations to the parallax and proper motion of a star. In a previous paper in this series, we argue that the astrometric mission Gaia can identify long-period binaries by precisely measuring these arcs. An arc in a star’s path can also be caused by a flyby: a hyperbolic encounter with another massive object. We quantify the apparent acceleration over time induced by a companion star as a function of the impact parameter, velocity of interaction, and companion mass. In principle, Gaia could be used to astrometrically identify the contribution of massive compact halo objects to the local dark matter potential of the Milky Way. However, after quantifying their rate and Gaia’s sensitivity, we find that flybys are so rare that Gaia will probably never observe one. Therefore, every star in the Gaia database exhibiting astrometric acceleration is likely in a long-period binary with another object. Nevertheless, we show how intermediate-mass black holes, if they exist in the local stellar neighborhood, can induce anomalously large accelerations on stars.