In eukaryotic cells, lipids in the form of triacylglycerols (TAGs) are the major reservoir of cellular carbon and energy. These TAGs are packed into specialized organelles called lipid droplets (LDs). They can be found in most, if not all, types of cells, from bacteria to human. Recent data suggest that rather than being simple storage organelles, LDs are very dynamic structures at the center of cellular metabolism. This is also true in plants and algae, where LDs have been implicated in many processes including energy supply; membrane structure, function, trafficking; and signal transduction. Plant and algal LDs also play a vital role in human life, providing multiple sources of food and fuel. Thus, a lot of attention has been paid to metabolism and function of these organelles in recent years. This review summarizes the most recent advances on LDs degradation as a key process for TAGs release. While the initial knowledge on this process came from studies in oilseeds, the findings of the last decade revealed high complexity and specific mechanisms of LDs degradation in plants and algae. This includes identification of numerous novel proteins associated with LDs as well as a prominent role for autophagy in this process. This review outlines, systemizes, and discusses the most current data on LDs catabolism in plants and algae.