Direction-of-arrival (DOA) estimation in coprime array involves the problem that there is the coexistence of both uncorrelated signals and coherent signals in the multipath environment. An approach to estimate DOA of uncorrelated and coherent signals separately, based on the coprime linear array and coprime planar array, is proposed in this paper. The uncorrelated signals are estimated first, where the root multiple signal classification (root-MUSIC) is applied in the coprime linear array and unitary estimating signal parameters via rotational invariance techniques (Unitary-ESPRIT) is utilized in the coprime planar array. Those subspace algorithms are of low complexity compared with the spectral peak search method. We then eliminate the components of noises and uncorrelated signals and reconstruct a covariance matrix of coherent signals. Finally, we use the root-MUSIC or Unitary-ESPRIT to resolve the one-dimensional or two-dimensional DOAs of coherent signals, respectively. The simulation results demonstrate the computational efficiency and high accuracy of the proposed algorithm. The results also prove that this algorithm can estimate the number of signals more than that of subarray sensors and separate two signals from the same angle. INDEX TERMS Direction-of-arrival, coprime linear array, coprime planar array, coherent signals.