Lipid droplets (LDs) are important cellular organelles due to their ability to accumulate and store lipids. LD dynamics are associated with various cellular and metabolic processes. Accurate monitoring of LD's size and shape is of prime importance as it indicates the metabolic status of the cells. Unintrusive continuous quantification techniques have a clear advantage in analyzing LDs as they measure and monitor the cells' metabolic function and droplets over time. Here, we present a novel machine-learning-based method for LDs analysis by segmentation of phasecontrast images of differentiated adipocytes (in vitro) and adipose tissue (in vivo).We developed a new workflow based on the ImageJ waikato environment for knowledge analysis segmentation plugin, which provides an accurate, label-free, live single-cell, and organelle quantification of LD-related parameters. By applying the new method on differentiating 3T3-L1 cells, the size of LDs was analyzed over time in differentiated adipocytes and their correlation with other morphological parameters. Moreover, we analyzed the LDs dynamics during catabolic changes such as lipolysis and lipophagy and demonstrated its ability to identify different cellular subpopulations based on their structural, numerical, and spatial variability. This analysis was also implemented on unstained ex vivo adipose tissues to measure adipocyte size, an important readout of the tissue's metabolism. The presented approach can be applied in different LD-related metabolic conditions to provide a better understanding of LD biogenesis and function in vivo and in vitro while serving as a new platform that enables rapid and accurate screening of data sets.