Castor bean (Ricinus communis L.) originated in East Africa and then diffused to warm-temperate, subtropical, and tropical regions of the world. The high lipid content in the castor beans is extracted for use in pharmaceutical and industrial applications. The castor oil lipid profile is naturally composed of 90% ricinoleic acid and the remaining 10% is mainly composed of linoleic, oleic, stearic, and linolenic fatty acids. The highly toxic compound ricin within the seeds is insoluble in oil, making castor oil free from this toxin and safe to use for industrial and cosmetic applications. Among the main uses of castor oil are reported industrial uses such as component for lubricants, paints, coatings, polymers, emulsifiers, cosmetics, and medicinal uses as a laxative. There is also significant commercial potential for utilization of the whole castor bean plant such as animal feed, fertilizer, biofuel, and also for phytoremediation. Several breeding programs have been planned to improve the castor’s characteristics needed for its current or potential uses. In this review, after summarizing data on castor bean agronomy and uses, we focus on the main advances in Castor bean classical and biotechnological breeding programs, underlining the high potential of this oil crop. In particular, the main challenges of castor breeding programs are to increase yield, mainly through the selection of growth habits allowing mechanized harvest, and beneficial compound content, mainly the oil, and to decrease the toxic compounds content, mainly ricin.