In this work, we performed static density functional theory calculations and ab initio metadynamics simulations to systematically investigate the association mechanisms and dynamic structures of four kinds of ion pairs that could be formed before the nucleation of CaCO3. For Ca2+–HCO3− and Ca2+–CO32− pairs, the arrangement of ligands around Ca2+ evolves between the six-coordinated octahedral structure and the seven-coordinated pentagonal bipyramidal structure. The formation of ion pairs follows an associative ligand substitution mechanism. Compared with HCO3−, CO32− exhibits a stronger affinity to Ca2+, leading to the formation of a more stable precursor phase in the prenucleation stage, which promotes the subsequent CaCO3 nucleation. In alkaline environments, excessive OH− ions decrease the coordination preference of Ca2+. In this case, the formation of Ca(OH)+–CO32− and Ca(OH)2–CO32− pairs favors the dissociative ligand substitution mechanism. The inhibiting effects of OH− ion on the CaCO3 association can be interpreted from two aspects, i.e., (1) OH− neutralizes positive charges on Ca2+, decreases the electrostatic interactions between Ca2+ and CO32−, and thus hinders the formation of the CaCO3 monomer, and (2) OH− decreases the capacity of Ca2+ for accommodating O, making it easier to separate Ca2+ and CO32− ions. Our findings on the ion association behaviors in the initial stage of CaCO3 formation not only help scientists evaluate the impact of ocean acidification on biomineralization but also provide theoretical support for the discovery and development of more effective approaches to manage undesirable scaling issues.