The primary and ambient signal decomposition of a stereo sound is a key step to the stereo upmix. The principal component analysis (PCA) is one of the most widely used methods of primary-ambient signal decomposition. However, previous PCA-based decomposition algorithms assume that stereo sound sources are only amplitude-panned without any consideration of phase difference. So it occurs some performance degradation in case of live recorded stereo sound. In this paper, we propose a new PCA-based stereo decomposition algorithm that can consider the phase difference between the channel signals. The proposed algorithm overcomes limitation of conventional signal model using PCA with phase alignment. The phase alignment is realized by using inter-channel phase difference (IPD) which is widely used in parametric stereo coding. Moreover, Enhanced Modified PCA(EMPCA) is combined to solve the problem of conventional PCA caused by Primary to Ambient energy Ratio(PAR) and panning angle dependency. The simulation results are presented to show the improvements of the proposed algorithm.