Diffusion-weighted imaging holds the potential to improve the diagnosis and biological characterization of pancreatic disease, and in particular pancreatic cancer, which exhibits decreased values of the apparent diffusion coefficient (ADC). Yet, variable and overlapping ADC values have been reported for the healthy and the pathological pancreas, including for cancer and other benign conditions. This controversy reflects the complexity of probing the water-diffusion process in the pancreas, which is dependent upon multiple biological factors within this organ's unique physiological environment. In recent years, extensive studies have investigated the correlation between tissue properties including cellularity, vascularity, fibrosis, secretion and microstructure and pancreatic diffusivity. Understanding how the various physiological and pathological features and the underlying functional processes affect the diffusion measurement may serve to optimize the method for improved diagnostic gain. Therefore, the aim of the present review article is to elucidate the relationship between pancreatic tissue characteristics and diffusion MRI measurement.