Material decomposition (MD) is an important application of computer tomography (CT). For phase contrast imaging, conventional MD methods are categorized into two types with respect to different operation sequences, i.e., “before” or “after” image reconstruction. Both categories come down to two-step methods, which have the problem of noise amplification. In this study, we incorporate both phase and absorption (PA) information into MD process, and correspondingly develop a simultaneous algebraic reconstruction technique (SART). The proposed method is referred to as phase & absorption material decomposition-SART (PAMD-SART). By iteratively solving an optimization problem, material composition and substance quantification are reconstructed directly from absorption and differential phase projections. Comparing with two-step MD, the proposed one-step method is superior in noise suppression and accurate decomposition. Numerical simulations and synchrotron radiation based experiments show that PAMD-SART outperforms the classical MD method (image-based and dual-energy CT iterative method), especially for the quantitative accuracy of material equivalent atomic number.