Simultaneously non-destructive, high resolution, and label-free imaging are of paramount importance for studies of complex biological systems, from viruses to cell cultures. Electron imaging techniques achieve extreme resolution but require slicing the sample to obtain volumetric information. On the other hand, X-rays’ high penetrative ability combined with cryogenic temperatures allows access to high resolution while preserving the sample’s structure. However, both X-ray and electron techniques do not currently allow label-free imaging with tissue specificity. Here, we combine a polarimetric approach with coherent diffractive imaging to reveal new ways to overcome this by mapping variations of anisotropy in the complex refractive index of cellular structures to differentiate between various tissues without chemical labeling. In this article, we demonstrate imaging of cancer-associated fibroblasts using birefringent coherent diffractive imaging with enhanced sensitivity to fibrous structures and their orientation as well as the possibility to differentiate the nucleus of the cell. We also propose a modeled soft X-ray experiment on the SARS-CoV-2 virus to address the possibility of leveraging the polarimetric birefringent contrast to spatially resolve the dynamical interaction of the virus with its host environment. We hope that our approach can open up avenues in the future to map and understand how SARS viruses bind with human epithelial cells.