Background Cancer has great implications for psychological, social, economic, and emotional dimensions. Psychological distress is overwhelming among cancer patients following a confirmed diagnosis. However, little is known about the prevalence of psychological distress and associated factors among cancer patients in Africa Sub-Saharan. Thus, this study aimed to assess the prevalence of psychological distress and associated factors among cancer patients in public hospitals in Addis Ababa, Ethiopia. Methods An institution-based cross-sectional study was conducted among cancer patients from September 15, 2019, to June 30, 2020. A total of 386 cancer patients selected through a simple random sampling technique participated in the study. Data were collected by an interview-administered questionnaire to evaluate psychological distress with a distress thermometer and social support with the Oslo 3-items Social Support Scale. The collected data were entered into Epi-data version 4.2 and exported into SPSS 25 for analysis, and then binary and multivariate logistic regressions were done to identify the association between dependent and independent variables. Results A total of 386 study participants were included in the study with a response rate of 91.4%. The prevalence of psychological distress among cancer patients in public hospitals in Addis Ababa, Ethiopia was 64.5%. Age > 45 years [AOR = 0.41; 95% CI (0.22–0.77)], marital status of being divorced [AOR = 3.3; 95%CI (1.23–8.71)] and married [AOR = 3.2; 95%CI (1.03–10.40)], rural residence [AOR = 1.5; 95%CI (1.15–5.18)], cancer stage II [AOR = 3.9; 95%CI (1.90–15.50)], stage III [AOR = 3.5;95%CI (1.45–8.44)] and stage IV [AOR = 3.4; 95%CI (1.90–10.11)], co-morbidity [AOR = 0.07; 95%CI: (0.03–0.17)], and moderate social support [AOR = 0.36; 95%CI (0.14–0.60)] and strong social support [AOR = 0.06; 95%CI (0.03–0.12)] were found to be significantly associated with psychological distress. Conclusion The prevalence of psychological distress among cancer patients in public hospitals in Addis Ababa, Ethiopia was high, and age, marital status, place of residence, cancer stage, co-morbidity, and social support were associated with psychological distress. Therefore, interventions focusing on these findings require special emphasis during designing interventions aimed at decreasing psychological distress.