Direction-of-arrival (DOA) estimation in multipath environment is an important issue for passive bistatic radar (PBR) using frequency agile phased array VHF radar as illuminator of opportunity. Under such scenario, the main focus of this paper is to cope with the closely spaced uncorrelated and coherent signals in low signal-to-noise ratio and limited snapshots. Making full use of the characteristics of moduli of eigenvalues, the DOAs of the uncorrelated signals are firstly estimated. Afterwards, their contributions are eliminated by means of spatial difference technique. Finally, in order to improve resolution and accuracy DOA estimation of remaining coherent signals while avoiding the cross-terms effect, a new beamforming solution based iterative adaptive approach (IAA) is proposed to deal with a reconstructed covariance matrix. The proposed method combines the advantages of both spatial difference method and the IAA algorithm while avoiding their shortcomings. Simulation results validate its effectiveness; meanwhile, the good performances of the proposed method in terms of resolution probability, detection probability, and estimation accuracy are demonstrated by comparison with the existing methods.