Reading requires efficient communication between brain regions that are situated all over the cortex. These brain areas are structurally connected by white-matter pathways that develop over the period of reading acquisition. Here, we conducted a cross-sectional study to assess the relationship between white-matter tracts and reading performance across the lifespan. Methods: Behavioral (reading performance) and neuroimaging (diffusion tensor imaging [DTI]) data were collected from participants aged 3–21 years as a part of the multi-site project called the pediatric imaging neurocognition genetics study. DTI measures (fractional anisotropy [FA], mean diffusivity, radial diffusivity and axial diffusivity) of bilateral dorsal tracts (arcuate fasciculus, superior longitudinal fasciculus, and posterior superior longitudinal fasciculus) and ventral tracts (uncinate fasciculus, inferior fronto-occipital fasciculus, and inferior longitudinal fasciculus) were extracted. Reading performance was calculated as the number of items correctly read (words for older children and letters in the case of young children). Correlational and regression analyses were conducted between the DTI measures and reading scores. Results: During the early stages of reading acquisition (ages 3–6), the dorsal tracts were positively related to reading performance (as FA goes up, reading performance goes up). For ages 6–10, the dorsal tracts remain positively associated with reading performance and a relationship between ventral tracts and reading performance emerges. From the age of 10 onwards, the dorsal tracts no longer correlate with reading performance, and a brain-behavior relationship in the right ventral tracts begins to shift, whereby higher FA is associated with lower reading performance. In addition to the involvement of left hemispheric tracts, this study revealed the initial engagement of right hemispheric tracts during the early stages of reading acquisition. Conclusion: We discuss these different associations of white-matter tracts with reading during development in the context of the biological processes model of myelination and pruning.