The design of a direct time-of-flight complementary metal-oxide-semiconductor (CMOS) image sensor (dToF-CIS) based on a single-photon avalanche-diode (SPAD) array with an in-pixel time-to-digital converter (TDC) must contemplate system-level aspects that affect its overall performance. This paper provides a detailed analysis of the impact of process parameters, voltage supply, and temperature (PVT) variations on the time bin of the TDC array. Moreover, the design and characterization of a global compensation loop is presented. It is based on a phase locked loop (PLL) that is integrated on-chip. The main building block of the PLL is a voltage-controlled ring-oscillator (VCRO) that is identical to the ones employed for the in-pixel TDCs. The reference voltage that drives the master VCRO is distributed to the voltage control inputs of the slave VCROs such that their multiphase outputs become invariant to PVT changes. These outputs act as time interpolators for the TDCs. Therefore the compensation scheme prevents the time bin of the TDCs from drifting over time due to the aforementioned factors. Moreover, the same scheme is used to program different time resolutions of the direct time-of-flight (ToF) imager aimed at 3D ranging or depth map imaging. Experimental results that validate the analysis are provided as well. The compensation loop proves to be remarkably effective. The spreading of the TDCs time bin is lowered from: (i) 20% down to 2.4% while the temperature ranges from 0 °C to 100 °C; (ii) 27% down to 0.27%, when the voltage supply changes within ±10% of the nominal value; (iii) 5.2 ps to 2 ps standard deviation over 30 sample chips, due to process parameters’ variation.