Sheep and goats are an important commodity for smallholder farmers across East Africa, but severe limitations remain in small ruminant production. This review aimed to identify specific constraints to small ruminant production and identify practical and sustainable solutions. From 54 eligible articles, most were focused in Ethiopia (n = 44) with only 6 studies performed in Tanzania and 4 in Uganda. The most frequently identified constraint in Ethiopia and Tanzania was disease (n = 28 and n = 3, respectively), and in Uganda, it was the lack of access to veterinary services (n = 4). Additionally, access to good breeding stock, lack of animal records, and an established marketing chain were also mentioned in all the three countries. Ectoparasites, gastrointestinal parasites, orf, and sheep/goat pox were the most frequently mentioned disease challenges causing productivity losses. Many articles provided potential solutions as suggested by farmers, including improved access to veterinary services and medicines, improved record keeping, and access to good breeding stock. Farmers highlighted the value of community-based participatory development plans to increase education on disease control, land management, and husbandry. This review also highlighted knowledge gaps, the need for further research, particularly in Tanzania and Uganda, and the importance of addressing multiple challenges holistically due to the links between constraints.