Background Late diagnosis of prostate cancer is common in Uganda and elsewhere. Diagnosis in advanced stages is associated with high mortality, morbidity and low quality of life. We estimated time taken from perception of symptoms attributable to prostate cancer to biopsy among patients with prostate cancer at the Uganda Cancer Institute (UCI) and the associated factors. Methods We conducted a retrospective cohort analysis of records of 280 patients with histologically confirmed diagnosis of prostate cancer at UCI from January 2016 to December 2017. Time to diagnosis was obtained from the difference between approximate date of onset of initial symptoms and date when a biopsy was taken. Late diagnosis was that when an individual was diagnosed with prostate cancer stage III or IV whereas stages I and II were classified as early. We used modified Poisson regression to assess factors associated with timing of diagnosis among the patients. Results The median time from first perceived symptoms to biopsy for prostate cancer patients was 12 (IQR 5-24) months and 76% were diagnosed after 4 months of symptoms. Median age at time of diagnosis of patients was 70 (IQR66-74.5) years and at least 50% were aged between 65-74 years. About 81.8% of patients were diagnosed late; of which 35.7% were in stage III and 46.1% were in stage IV. Nearly all patients presented with raised prostate specific antigen with a median prostate specific antigen of 100.2 (IQR 36.02-350) ng/ml of blood at the time of admission. In adjusted analysis, patients whose biopsies were taken before 5 months of recognising symptoms were two times as likely to have cancer stage I and II compared to those patients in whom the biopsies were taken after 4 months. Conclusion More than three in four patients were diagnosed late. Taking a biopsy after 4 months of initiation of symptoms was partially responsible for the delay. To improve time to diagnosis, communities should be educated about symptoms of prostate cancer and advised to seek health care early. Health care workers should be sensitised to suspect prostate cancer among patients to allow timely referral for specialised assessment and management.