Due to sensor characteristics, geographical environment, electromagnetic interference, electromagnetic silence, information countermeasures, and other reasons, there may be significant system errors in sensors in multi-sensor tracking systems, resulting in poor track-to-track association (TTTA) effect of the system. In order to solve the problem of TTTA under large system errors, this paper proposes an asynchronous anti-bias TTTA algorithm that utilizes the average distance between the nearest neighbor intervals between tracks. This algorithm proposes a systematic error interval processing method to track coordinates, and then defines the nearest neighbor interval average distance between interval coordinate datasets and interval coordinate points, and then uses grey theory to calculate the correlation degree between tracks. Finally, the Jonker–Volgenant algorithm is combined to use the canonical allocation method for TTTA judgment. The algorithm requires less prior information and does not require error registration. The simulation results show that the algorithm can ensure a high average correct association rate (over 98%) of asynchronous unequal rate tracks under large system errors, and achieve stable association, with good association and anti-bias performance. Compared with other algorithms, the algorithm maintains good performance for different target numbers and processing cycles, and has good superiority and robustness.