The vascular risk associated with obesity is particularly associated with visceral adiposity, but recent studies suggest that ectopic fat might contribute to the increased risk of atherosclerotic cardiovascular disease. Our study aimed to explore the connection between arterial calcification of the aorta and renal arteries with visceral and ectopic fat deposits, including liver, pancreatic, and renal sinus fat. Retrospective analysis of thoracoabdominal multi-slice computed tomography (MSCT) scans of 302 patients included measurements of calcification volumes of thoracic and abdominal aorta, and of both renal arteries. On the same scans, the visceral fat volume, liver-to-spleen ratio, pancreatic-to-spleen ratio, and both renal sinus fat areas were retrieved. Logistic regression showed the left kidney sinus fat area to be the most strongly associated with calcifications in the aorta and both renal arteries (coef. from 0.578 to 0.913, p < 0.05). The visceral fat positively predicted aortic calcification (coef. = 0.462, p = 0.008), and on the contrary, the pancreatic fat accumulation even showed protective effects on thoracic and abdominal aorta calcification (coef. = −0.611 and −0.761, p < 0.001, respectively). The results suggest that ectopic fat locations differently impact the calcification of arteries, which should be further explored.