Knowledge of the nitrogen-vacancy center formation kinetics in diamond is critical to engineering sensors and quantum information devices based on this defect. Here we utilize the longitudinal tracking of single NV centers to elucidate NV defect kinetics during high-temperature annealing from 800-1100 • C in high-purity chemical-vapor-deposition diamond. We observe three phenomena which can coexist: NV formation, NV quenching, and NV orientation changes. Of relevance to NV-based applications, a 6 to 24-fold enhancement in the NV density, in the absence of sample irradiation, is observed by annealing at 980 • C, and NV orientation changes are observed at 1050 • C. With respect to the fundamental understanding of defect kinetics in ultra-pure diamond, our results indicate a significant vacancy source can be activated for NV creation between 950-980 • C and suggests that native hydrogen from NVHy complexes plays a dominant role in NV quenching, in agreement with recent ab initio calculations. Finally, the direct observation of orientation changes allows us to estimate an NV diffusion barrier of 5.1 eV. arXiv:1907.07793v1 [cond-mat.mtrl-sci]