The tropics produce a range of fruit from tree crops that cannot be grown in colder climates. Bananas, mangos, several nuts, spices, coffee, and cacao are widely traded and much sought after around the world. However, the sustainable production of these tropical tree fruit crops faces significant challenges. Among these, losses due to pests and diseases play a large part in reducing yields, quality, and profitability. Using bananas and cacao as key examples, we outline some of the reasons fungal and oomycete diseases cause such significant losses to tropical tree crops. Cultivation of monocultures derived from limited genetic diversity, environmental conditions conducive for disease development, high levels of disease incidence and severity, a lack of disease resistance in planting materials, shortages of labor, and inadequate infrastructure and investment pose significant challenges, especially for smallholder producers. The expansion of travel and trade has given rise to emerging infectious plant diseases that add further insecurity and pressure. We conclude that holistic actions are needed on multiple fronts to address the growing problem of disease in tropical fruit tree crops.