Cooperative navigation aims at improving positioning accuracy of Autonomous Underwater Vehicles (AUVs). In this paper, a dual leaders cooperative navigation method is proposed based on Cross Entropy (CE) algorithm. Since the trajectories of the slave AUVs are assumed to be predetermined, the Markov Decision Process (MDP) is also integrated in the proposed algorithm to generate optimal trajectories of master AUVs from the perspective of probability. Firstly, the navigation model and cost functions are established for the cooperative navigation system with multiple masters and slaves. Then, the CE algorithm is used to train the system with help of MDP to obtain the path of the master AUVs. In the simulation, the cooperative localization trajectories of the slave AUVs are obtained by Extended Kalman Filter (EKF) and are compared with other positioning methods. The results show that the trajectories of dual master AUVs obtained by the proposed algorithm can not only reduce the observation error of the slave AUVs in the system effectively, but also keep relative measurement distance between the master AUVs and the slave AUVs in a suitable range. INDEX TERMS Cooperative navigation, cross entropy, Markov decision process, path planning.