The demand for rice in Eastern and Southern Africa is rapidly increasing because of changes in consumer preferences and urbanization. However, local rice production lags behind consumption, mainly due to low yield levels. In order to set priorities for research and development aimed at improving rice productivity, there is a need to characterize the rice production environments, to quantify rice yield gaps—that is, the difference between average on‐farm yield and the best farmers’ yield—and to identify causes of yield gaps. Such information will help identifying and targeting technologies to alleviate the main constraints, and consequently to reduce existing yield gaps. Yield gap surveys were conducted on 357 rice farms at eight sites (19–50 farmers per site) across five rice‐producing countries in Eastern and Southern Africa—that is Ethiopia, Madagascar, Rwanda, Tanzania and Uganda—for one or two years (2012–13) to collect both quantitative and qualitative data at field and farm level. Average farm yields measured at the eight sites ranged from 1.8 to 4.3 t/ha and the average yield gap ranged from 0.8 to 3.4 t/ha. Across rice‐growing environments, major causes for yield variability were straw management, weeding frequency, growth duration of the variety, weed cover, fertilizer (mineral and organic) application frequency, levelling and iron toxicity. Land levelling increased the yield by 0.74 t/ha, bird control increased the yield by 1.44 t/ha, and sub‐optimal management of weeds reduced the yield by 3.6 to 4.4 t/ha. There is great potential to reduce the current rice yield gap in ESA, by focusing on improvements of those crop management practices that address the main site‐specific causes for sub‐optimal yields.