The directional polarimetric camera (DPC) is a Chinese satellite sensor with a large field of view (FOV) (±50° both along-track and cross-track) and a high spatial resolution (about 3.3 km at nadir) that operates in a sun-synchronous orbit. It is a difficult task to calibrate the in-flight relative radiometric variation of the sensors with such a wide FOV. In this study, a new method based on Rayleigh scattering over the ocean is developed to estimate the radiometric sensitivity variation over the whole FOV of DPC. Firstly, the theoretical uncertainty of the method is analyzed to calibrate the relative radiometric response. The calibration uncertainties are about 2–6.9% (depending on the wavelength) when the view zenith angle (VZA) is 0° and decrease to about 1–3.8% when VZA increases to 70°. Then, the method is applied to evaluate the long-term radiometric drift of the DPC. It is found that the radiometric response of DPC/GaoFen-5 over the whole FOV is progressively drifting over time. The sensitivity at shorter bands decreases more strongly than longer bands, and at the central part of the optics decreases more strongly than the marginal part. During the 14 months (from March 2019 to April 2020) of operational running in-orbit, the DPC radiometric responses of 443 nm, 490 nm, 565 nm, and 670 nm bands drifted by about 4.44–23.08%, 4.75–16.22%, 3.86–9.81%, and 4.7–16.86%, respectively, from the marginal to the central part of the FOV. The radiometric sensitivity has become more stable since January 2020. The monthly radiometric drift is separated into the relative radiometric part and the absolute radiometric part. The relative radiometric drift of DPC is found to be smoothly varying with VZA, which can be parameterized as a polynomial function via VZA. At last, the temporal radiometric drift of DPC/GaoFen-5 is corrected by combining the relative and absolute radiometric coefficients. The correction is convincing by cross calibration with MODIS/Aqua observation over the desert sites and improving the aerosol retrievals. The Rayleigh method in this study is efficient for the radiometric sensitivity calibration of wide FOV satellite sensors.