The receptor-binding domain (RBD) of the SARS-CoV-2 spike protein (RBDCoV2) has a higher binding affinity to the human receptor angiotensin-converting enzyme 2 (ACE2) than the SARS-CoV RBD (RBDCoV). Here, we performed molecular dynamics (MD) simulations, binding free energy (BFE) calculations, and interface residue contact network (IRCN) analysis to explore the mechanistic origin of different ACE2-binding affinities of the two RBDs. The results demonstrate that, when compared to the RBDCoV2-ACE2 complex, RBDCoV-ACE2 features enhanced dynamicsand inter-protein positional movements and increased conformational entropy and conformational diversity. Although the inter-protein electrostatic attractive interactions are the primary determinant for the high ACE2-binding affinities of both RBDs, the significantly enhanced electrostatic attractive interactions between ACE2 and RBDCoV2 determine the higher ACE2-binding affinity of RBDCoV2 than of RBDCoV. Comprehensive comparative analyses of the residue BFE components and IRCNs between the two complexes reveal that it is the residue changes at the RBD interface that lead to the overall stronger inter-protein electrostatic attractive force in RBDCoV2-ACE2, which not only tightens the interface packing and suppresses the dynamics of RBDCoV2-ACE2, but also enhances the ACE2-binding affinity of RBDCoV2. Since the RBD residue changes involving gain/loss of the positively/negatively charged residues can greatly enhance the binding affinity, special attention should be paid to the SARS-CoV-2 variants carrying such mutations, particularly those near or at the binding interfaces with the potential to form hydrogen bonds and/or salt bridges with ACE2.