Background: Low skeletal muscle mass is associated with increased postoperative morbidity and worse survival following resection for perihilar cholangiocarcinoma (PHC). We investigated the predictive value of skeletal muscle mass and density for overall survival (OS) of all patients with suspected PHC, regardless of treatment. Methods: Baseline characteristics and parameters regarding disease and treatment were collected from all patients with PHC from 2002 to 2014. Skeletal muscle mass and density were measured at the level of the third lumbar vertebra on CT. The association between skeletal muscle mass and density with OS was investigated using the Kaplan-Meier method and Cox survival. Results: Median OS in 233 included patients did not differ between those with and without low skeletal muscle mass (p = 0.203), whereas a significantly different median OS (months) was observed between patients with low (HR 7.0, 95% CI 4.7–9.3) and high (HR 12.1, 95% CI 8.1–16.1) skeletal muscle density (p = 0.004). Low skeletal muscle density was independently associated with decreased OS (HR 1.78, 95% CI 1.03–3.07, p = 0.040) within the first 6 months but not after 6 months (HR 0.68, 95% CI 0.44–1.07, p = 0.093), after adjusting for age, tumour size and suspected peritoneal or other distant metastases on imaging. Conclusion: A time-dependent effect of skeletal muscle density on OS was found in patients with PHC, regardless of subsequent treatment. Low skeletal muscle density may identify patients at risk for early death.