We investigate interferometric techniques to estimate the deflection angle of an optical beam and compare them to the direct detection of the beam deflection. We show that quantum metrology methods lead to a unifying treatment for both single photons and classical fields. Using the Fisher information to assess the precision limits of the interferometric schemes, we show that the precision can be increased by exploiting the initial transverse displacement of the beam. This gain, which is present for both Sagnac and Mach-Zehnder-like configurations, can be considerable when compared to noninterferometric methods. In addition to the fundamental increase in precision, the interferometric schemes have the technical advantage that (i) the precision limits can be saturated by a sole polarization measurement on the field, and that (ii) the detection system can be placed at any longitudinal position along the beam. We also consider position-dependent polarization measurements, and show that in this case the precision increases with the propagation distance, as well as the initial transverse displacement.