We present single-shot quantitative phase imaging with polarization differential interference contrast (PDIC) for a differential interference contrast microscope which records the unfiltered Stokes vector of the differential interference pattern with a polarization camera. PDIC enables high spatial resolution phase imaging in real-time, applicable to either absorptive or transparent samples and integrates simply with epi-fluorescence imaging. An algorithm with total variation regularization is further introduced to solve the quantitative phase map from the partial derivative along one single axis, improving the accuracy and the image quality from the Fourier transform solution. After quantifying the accuracy of PDIC phase imaging with numerical simulations and phantom measurements, we demonstrate the biomedical applications by imaging the quantitative phase of both stained and unstained histological tissue sections and visualizing the fission yeast Schizosaccharomyces pombe's cytokinesis.