Reducing the moment of inertia improves the sensitivity of a mechanically based torque sensor, the parallel of reducing the mass of a force sensor, yet the correspondingly small displacements can be difficult to measure. To resolve this, we incorporate cavity optomechanics, which involves co-localizing an optical and mechanical resonance. With the resulting enhanced readout, cavity-optomechanical torque sensors are now limited only by thermal noise. Further progress requires thermalizing such sensors to low temperatures, where sensitivity limitations are instead imposed by quantum noise. Here, by cooling a cavity-optomechanical torque sensor to 25 mK, we demonstrate a torque sensitivity of 2.9 yNm/. At just over a factor of ten above its quantum-limited sensitivity, such cryogenic optomechanical torque sensors will enable both static and dynamic measurements of integrated samples at the level of a few hundred spins.