Among the most important processes in the self-assembly of the eukaryotic cell is the synthesis of its organelles, specialized biochemical compartments that house processes crucial to cellular physiology. A critical property that governs organelle function is its size. Numerous molecular factors that regulate the sizes of a diverse array of organelles, including the Golgi, mitochondria, peroxisomes and lipid droplets among others, have been identified. However, our understanding of the quantitative principles governing organelle size control remains incomplete. Here, we combine organelle size data from the single-celled eukaryote Saccharomyces cerevisiae and mathematical theory to show that cells can robustly control organelle size fluctuations across a range of organelle sizes. In particular, our framework suggests that organelle size increases in random bursts from a limited pool of building blocks. Bursty organelle growth allows the cell to decouple the average magnitude of organelle size fluctuations from mean organelle size, provided the bursts do not deplete the pool of building blocks from which organelles grow. Bursty growth thus provides a potentially general mechanism by which cells can regulate the noise of the sizes of its subcellular structures.