Cerebral ischemia is one of the major diseases associated with death or disability among patients. To date, there is a lack of effective treatments, with the exception of thrombolytic therapy that can be administered during the acute phase of ischemic stroke. Cerebral ischemia can cause a variety of pathological changes, including microvascular basal membrane matrix, endothelial cell activation, and astrocyte adhesion, which may affect signal transduction between the microvessels and neurons. Therefore, researchers put forward the concept of neurovascular unit, including neurons, axons, astrocytes, microvasculature (including endothelial cells, basal membrane matrix, and pericyte), and oligodendrocytes. Numerous studies have demonstrated that exercise can produce protective effects in cerebral ischemia, and that exercise may protect the integrity of the blood-brain barrier, promote neovascularization, reduce neuronal apoptosis, and eventually lead to an improvement in neurological function after cerebral ischemia. In this review, we summarized the potential mechanisms on the effect of exercise on cerebral ischemia, by mainly focusing on the neurovascular unit, with the aim of providing a novel therapeutic strategy for future treatment of cerebral ischemia.