The Golgi apparatus plays a central role in cell homeostasis, not only in processing and maturing newly synthesized proteins and lipids but also in orchestrating their sorting, packing, routing and recycling on the way to their final destination. These multiple secretory pathways require a complex ballet of vesicular and tubular carriers that continuously bud off from donor membranes and fuse to acceptor membranes. Membrane trafficking is particularly prominent in axons, where cargo molecules have a long way to travel before they reach the synapse, and in oligodendrocytes, which require an immense increase in membrane surface in order to sheathe axons in myelin. Interestingly, in recent years, genes encoding Golgi-associated proteins with a role in membrane trafficking have been found to be defective in an increasing number of inherited disorders whose clinical manifestations include postnatal-onset microcephaly (POM), white matter defects and intellectual disability. Several of these genes encode RAB GTPases, RAB-effectors or RAB-regulating proteins, linking POM and intellectual disability to RAB-dependent Golgi trafficking pathways and suggesting that their regulation is critical to postnatal brain maturation and function. Here, we review the key roles of the Golgi apparatus in post-mitotic neurons and the oligodendrocytes that myelinate them, and provide an overview of these Golgi-associated POM-causing genes, their function in Golgi organization and trafficking and the likely mechanisms that may link dysfunctions in RAB-dependent regulatory pathways with POM.