During the last two decades, large epidemiological studies have shown that the physical environment, including noise, air pollution or heavy metals, have a considerable impact on human health. It is known that the most common cardiovascular risk factors are all associated with endothelial dysfunction. Vascular tone, circulation of blood cells, inflammation, and platelet activity are some of the most essential functions regulated by the endothelium that suffer negative effects as a consequence of environmental pollution, causing endothelial dysfunction. In this review, we delineate the impact of environmental risk factors in connection to endothelial function. On a mechanistic level, a significant number of studies suggest the involvement of endothelial dysfunction to fundamentally drive the adverse endothelium health effects of the different pollutants. We focus on well-established studies that demonstrate the negative effects on the endothelium, with a focus on air, noise, and heavy metal pollution. This in-depth review on endothelial dysfunction as a consequence of the physical environment aims to contribute to the associated research needs by evaluating current findings from human and animal studies. From a public health perspective, these findings may also help to reinforce efforts promoting the research for adequate promising biomarkers for cardiovascular diseases since endothelial function is considered a hallmark of environmental stressor health effects.