Calcium ATPase is a member of the P-type ATPase, and it pumps calcium ions from the cytoplasm into the reticulum against a concentration gradient. Several X-ray structures of different conformations have been solved in recent years, providing basis for elucidating the active transport mechanism of Ca2+ ions. In this work, molecular dynamics (MD) simulations were performed at atomic level to investigate the dynamical process of calcium ions moving from the outer mouth of the protein to their binding sites. Five initial locations of Ca2+ ions were considered, and the simulations lasted for 2 or 6 ns, respectively. Specific pathways leading to the binding sites and large structural rearrangements around binding sites caused by uptake of calcium ions were identified. A cooperative binding mechanism was observed from our simulation. Firstly, the first Ca2+ ion binds to site I, and then, the second Ca2+ ion approaches. The interactions between the second Ca2+ and the residues around site I disturb the binding state of site I and weaken its binding ability for the first bound Ca2+. Because of the electrostatic repulsion of the second Ca2+ and the electrostatic attraction of site II, the first bound Ca2+ shifts from site I to site II. Concertedly, the second Ca2+ binds to site I, forming a binding state with two Ca2+ ions, one at site I and the other at site II. Both of Glu908 and Asp800 coordinate with the two Ca2+ ions simultaneously during the concerted binding process, which is believed to be the hinge to achieve the concerted binding. In our simulations, four amino acid residues that serve as the channel to link the outer mouth and the binding sites during the binding process were recognized, namely Tyr837, Tyr763, Asn911, and Ser767. The analyses regarding the activity of the proteins via mutations of some key residues also supported our cooperative mechanism.