Background Ethiopia is a multilingual and multinational federation with Addis Ababa serving as both the capital city of Oromia regional state and the seat of the Ethiopian federal government. Nevertheless, only Amharic is considered as the working language of the city and federal offices, including hospitals. As a result, Afaan Oromoo-speaking patients may be facing language barriers in the healthcare settings in Addis Ababa. Language barriers have the capacity to affect patients’ experience of care and treatment outcomes. This study, hence, examined the impacts of language barriers on the healthcare access and quality for the Afaan Oromoo-speaking patients in public hospitals in Addis Ababa. Methods In-depth interviews with patients (N = 27) and key informant interviews with healthcare providers (N = 9) were conducted in six public hospitals found in Addis Ababa. All the interviews were audio-taped and transcribed verbatim. A thematic analysis technique was employed to address the study objectives. Results The study participants indicated the widely existing problem of language discordance between patients and healthcare providers. The impacts of language barriers on the patients include preventable medical errors, low treatment adherence, low health-seeking behavior, additional treatment cost, increased length of hospital stays, weak therapeutic relation, social desirability bias, less confidence, and dissatisfaction with the healthcare. For the healthcare providers, language barriers are affecting their ability to take patient history, perform diagnoses and provide treatment, and have also increased their work burden. The use of ad hoc interpreters sourced from bilingual/multilingual patients, patient attendants, volunteer healthcare providers, and other casual people has been reported to deal with the problem of language barriers. Conclusion A significant number of Afaan Oromoo-speaking patients are facing language barriers in accessing quality healthcare in public hospitals in Addis Ababa, and this constitutes structural violence. As a way out, making Afaan Oromoo an additional working language of the public hospitals in Addis Ababa, the assignment of professional interpreters, and a hiring system that promotes the recruitment of qualified multi-lingual healthcare providers are suggested.