Though statehood nature traced back to ancient times, modern Ethiopia came into being in the second half of the nineteenth century under the military expansion of King Menelik II. Since then subsequent political systems in the country have failed to recognise and accommodate the country’s ethnolinguistic diversity. However, in 1991 the new government constitutionally recognised and institutionally accommodated the country’s diversity. While solving old problems of ethnic inequality and injustice, ethnic federalism has created new problems of ethnic tensions and conflict across Ethiopia. The article aims at investigating how and why ethnic federalism ended up being a source of ethnic conflicts in Ethiopia and suggests the way out. The article employed a qualitative research design and methods of data analysis and interpretation. The finding of the study shows the notion and implementation of federalism has to be blamed, which left unclear administration boundary, overlooked cross-cutting variables, Majority versus minority and Titular versus Settler problem, politicised ethnicity by transformed cultural communities into political communities, produced mega ethnic syndrome within the Ethiopian society, for the country’s ethnic troubles and conflicts. Reforming the federal system and its constitutions is the way forwarded to reduce ethnic tensions and create an inclusive society in Ethiopia.