The ability to assemble nano- and micro- sized colloidal components into highly ordered configurations is often cited as the basis for developing advanced materials. However, the dynamics of stochastic grain boundary formation and motion have not been quantified, which limits the ability to control and anneal polycrystallinity in colloidal based materials. Here we use optical microscopy, Brownian Dynamic simulations, and a new dynamic analysis to study grain boundary motion in quasi-2D colloidal bicrystals formed within inhomogeneous AC electric fields. We introduce “low-dimensional” models using reaction coordinates for condensation and global order that capture first passage times between critical configurations at each applied voltage. The resulting models reveal that equal sized domains at a maximum misorientation angle show relaxation dominated by friction limited grain boundary diffusion; and in contrast, asymmetrically sized domains with less misorientation display much faster grain boundary migration due to significant thermodynamic driving forces. By quantifying such dynamics vs. compression (voltage), kinetic bottlenecks associated with slow grain boundary relaxation are understood, which can be used to guide the temporal assembly of defect-free single domain colloidal crystals.