How do growing bacterial colonies get their shapes? While colony morphogenesis is well-studied in 2D, many bacteria grow as large colonies in 3D environments, such as gels and tissues in the body, or soils, sediments, and subsurface porous media. Here, we describe a morphological instability exhibited by large colonies of bacteria growing in 3D. Using experiments in transparent 3D granular hydrogel matrices, we show that dense colonies of four different species of bacteria—Escherichia coli, Vibrio cholerae, Pseudomonas aeruginosa, and Komagataeibacter sucrofermentans—generically roughen as they consume nutrients and grow beyond a critical size, eventually adopting a characteristic branched, broccoli-like, self-affine morphology independent of variations in the cell type and environmental conditions. This behavior reflects a key difference between 2D and 3D colonies: while a 2D colony may access the nutrients needed for growth from the third dimension, a 3D colony inevitably becomes nutrient-limited in its interior, driving a transition to rough growth at its surface. We elucidate the onset of roughening using linear stability analysis and numerical simulations of a continuum model that treats the colony as an ‘active fluid’ whose dynamics are driven by nutrient-dependent cellular growth. We find that when all dimensions of the growing colony substantially exceed the nutrient penetration length, nutrient-limited growth drives a 3D morphological instability that recapitulates essential features of the experimental observations. Our work thus provides a framework to predict and control the organization of growing colonies—as well as other forms of growing active matter, such as tumors and engineered living materials—in 3D environments.