The major microtubule-organizing center (MTOC) in animal cells, the centrosome, comprises a pair of centrioles surrounded by pericentriolar material (PCM), which nucleates and anchors microtubules. Centrosome assembly depends on the interactions of PCM with centrioles, PCM self-association and dynein-mediated transport. Here, we show that if centrioles are lost due to PLK4 depletion or inhibition, PCM still forms a single centrally located MTOC when non-centrosomal microtubule minus end organization pathways are disabled. Acentriolar MTOC assembly depends on dynein-driven coalescence of PCM clusters with attached microtubule minus ends and requires γ-tubulin, pericentrin, CDK5RAP2 and ninein, but not NEDD1, CEP152 or CEP192. PCM self-assembly is inhibited by AKAP450-dependent PCM recruitment to the Golgi and by CAMSAP2-mediated microtubule minus end stabilization. However, if CAMSAP2 is linked to a minus-end-directed motor, a single MTOC containing PCM components can still form, and its organization depends on the presence of pericentrin. Our results reveal that the formation of a single central MTOC in interphase mammalian cells is not strictly centriole dependent but can be driven by self-organization of PCM and microtubule minus ends.