The bottom-up processing of visual information is strongly influenced by top-down signals, at least part of which is thought to be conveyed from the frontal cortex through the frontal eye field (FEF) and the lateral intraparietal area (LIP). Here we investigated the architecture of multisynaptic pathways from the frontal cortex to the middle temporal area (MT) of the dorsal visual stream and visual area 4 (V4) of the ventral visual stream in macaques. In the first series of experiments, the retrograde trans-synaptic tracer, rabies virus, was injected into MT or V4. Three days after rabies injections, the second-order (disynaptically connected) neuron labeling appeared in the ventral part of area 46 (area 46v), along with the first-order (monosynaptically connected) neuron labeling in FEF and LIP. In the MT-injection case, second-order neurons were also observed in the supplementary eye field (SEF). In the next series of experiments, double injections of two fluorescent dyes, fast blue and diamidino yellow, were made into MT and V4 to examine whether the frontal inputs are mediated by distinct or common neuronal populations. Virtually no double-labeled neurons were observed in FEF or LIP, indicating that separate neuronal populations mediate the frontal inputs to MT and V4. The present results define that the multisynaptic frontal input to V4 arises primarily from area 46v, whereas the input to MT arises from not only area 46v but also SEF, through distinct FEF and LIP neurons. Segregated pathways from the frontal cortex possibly carry the functionally diverse top-down signals to each visual stream.