The magnetic fields of low-mass stars are observed to be variable on decadal timescales, ranging in behaviour from cyclic to stochastic. The changing strength and geometry of the magnetic field should modify the efficiency of angular momentum loss by stellar winds, but this has not been well quantified. In we investigated the variability of the Sun, and calculated the time-varying angular momentum loss rate in the solar wind. In this work, we focus on four low-mass stars that have all had their surface magnetic fields mapped for multiple epochs. Using mass loss rates determined from astrospheric Lyman-α absorption, in conjunction with scaling relations from the MHD simulations of Finley & Matt (2018), we calculate the torque applied to each star by their magnetised stellar winds. The variability of the braking torque can be significant. For example, the largest torque for Eri is twice its decadal averaged value. This variation is comparable to that observed in the solar wind, when sparsely sampled. On average, the torques in our sample range from 0.5 − 1.5 times their average value. We compare these results to the torques of Matt et al. (2015), which use observed stellar rotation rates to infer the long-time averaged torque on stars. We find that our stellar wind torques are systematically lower than the long-time average values, by a factor of ∼ 3 − 30. Stellar wind variability appears unable to resolve this discrepancy, implying that there remain some problems with observed wind parameters, stellar wind models, or the long-term evolution models, which have yet to be understood.