Artificial spin ices are ideal frustrated model systems in which to explore or design emergent phenomena with unprecedented characterization of the constituent degrees of freedom. In square spin ice, violations of the ice rule are topological excitations essential to the kinetics of the system, providing an ideal test bed for studying the dynamics of such defects under varied quench rates. In this work we examine possible scenarios including the Kibble-Zurek mechanism and critical coarsening in colloidal square and hexagonal ice under quenches from a weakly interacting liquid state into a strongly interacting regime. As expected, for infinitely slow quenches, the system is free of defects such as monopoles, while for increasing quench rate, an increasing number of defects in the form of monopoles or grain boundaries remain in the sample. For square ice, we find regimes in which the defect population decreases as a power law with decreasing quench rate. A detailed scaling analysis shows that for a wide range of parameters, including quench rates that are accessible by experiments, the behavior is best described by critical coarsening rather than the Zurek-Kibble scenario if we assume that the equilibrium phase transition in this system is in the Ising universality class. The appearance of critical coarsening is likely due to the strong defect interactions in the colloidal ice system leading to relevant defect dynamics during the quench. For hexagonal ice we do not find evidence for a power-law decay in the defect density, which is consistent with the absence of an equilibrium phase transition in the hexagonal ice as well as a lack of critical coarsening.