Background The use of anti-retroviral therapy (ART) in management and prevention of HIV/AIDS epidemic is a globally accepted strategy. In Uganda, despite the efforts to increase uptake of ART, adherence remains a huge challenge. This study, therefore, was conducted to explore psychosocial factors which influenced non-adherence to ART among young adults in Eastern Uganda. Methods This was an exploratory qualitative study conducted at the ART clinic of Mbale Regional Referral Hospital. A total of 35 in-depth interviews with young adults who had defaulted from taking ART drugs were conducted. Also, five key-informant interviews were conducted among the healthcare workers in the hospital. Thematic analysis approach was followed to analyze the data using NVIVO software (version 11). Results Non-adherence to ART was perceived to result from poor social support, poor coping mechanisms, unpredictable and busy work schedules, poverty, incompatible religious beliefs and practices. The poor social support factors included poor caregiver support, non-disclosure of HIV status, stigma and discrimination while factors linked to poor coping mechanisms included alcohol and substance abuse, psychosocial stress, depression, forgetfulness and feelings of self-hatred. Poverty limits access to treatment and basic needs including transportation to health facilities. Long waiting time at the ART clinic was the healthcare system factor which was thought to cause non-adherence among young adults. Conclusion Non-adherence to ART at Mbale Regional Referral Hospital in Eastern Uganda is attributed to various psychosocial factors such as stigma and discrimination, mental health problems, work-related problems and low socio-economic status, religious beliefs and poor knowledge, long waiting time in ART clinic and poor family support. Psychosocial counselling should therefore target the perceived causes of ART non-adherence so as to improve the adherence /compliance to ART.