Background In resource-limited countries such as Ethiopia, tuberculosis is the major cause of morbidity and mortality among people living with the human immunodeficiency virus. In the era of antiretroviral therapy, the effect of tuberculosis on the survival of patients who are living with human immunodeficiency virus has been poorly understood in Ethiopia. Therefore, this study aimed to determine the effect of active tuberculosis on the survival of HIV-infected adult patients who initiated antiretroviral therapy in public hospitals in Eastern Ethiopia. Methods An institution-based retrospective cohort study was conducted among 566 participants from January 1, 2014, to June 30, 2018. The collected data were entered into EpiData version 3.1 before being exported to Stata version 14 for analysis. A Cox proportional hazard model was used to determine the effect of active tuberculosis on the survival of HIV-infected adult patients who initiated antiretroviral therapy, and a p-value less than 0.05 and a 95% confidence level were used to declare statistical significance. Result Of the 566 patients included in the study, 76 died. The mortality rate was 11.04 per 100 person-years in tuberculosis co-infected patients, while it was 2.52 per 100 person-years in non-tuberculosis co-infected patients. The patients with tuberculosis co-infection had a 2.19 times higher hazard of death (AHR: 2.19; 95% CI: 1.17, 4.12) compared to those without tuberculosis. Advanced clinical stage, low CD4+ cell count, and previous episodes of an opportunistic infection other than tuberculosis were found to be independent predictors of mortality. Conclusion Co-infection with tuberculosis at antiretroviral therapy initiation increases the hazard of death approximately two-fold. Hence, we recommend key organizations to enhance the region’s collaborative interventional and preventative strategies for TB and HIV.