Myelin sheaths, which insulate the axons and ensure saltatory conduction of the nerve impulse, are generated and maintained via largely uncharacterized mechanisms. Ermin is an oligodendrocyte‐specific protein associated with the cytoskeleton, but how it regulates cytoskeletal remodeling during oligodendrocyte differentiation and its role in myelin maintenance are not clear. To address this, we generated mice constitutively deficient for Ermn, the Ermin‐coding gene. We found that aged Ermn‐knockout mice exhibit an aberrant myelin architecture, with splitting of myelin layers, peeling of the myelin sheath from axons, and breakdown of myelinated fibers. As a result, these mice had remarkably impaired motor coordination. Ermn knockout also accelerated cuprizone‐induced demyelination and exacerbated the associated movement disorders. Ermin was found to contribute to oligodendrocyte morphogenesis by associating with the myosin phosphatase Rho interacting protein (Mprip/p116RIP) and inactivating RhoA, a GTPase that controls cytoskeletal rearrangement in differentiating cells. These findings provide novel insights into the mechanisms regulating oligodendroglial differentiation, the maintenance of the myelin sheaths, and remyelination.