Understanding the process by which the cerebral hemispheres reach their mature functional organization remains challenging. We propose a theoretical account in which, in the domain of vision, faces and words come to be represented adjacent to retinotopic cortex by virtue of the need to discriminate among homogeneous exemplars. Orthographic representations are further constrained to be proximal to typically left--lateralized language--related information to minimize connectivity length between visual and language areas. As reading is acquired, orthography comes to rely more heavily (albeit not exclusively) on the left fusiform region to bridge between vision and language. Consequently, due to competition from emerging word representations, face representations that were initially bilateral become lateralized to the right fusiform region (albeit, again, not exclusively). We review recent research that describes constraints that give rise to this graded hemispheric arrangement. We then summarize empirical evidence from a variety of studies (behavioral, evoked response potential, functional imaging) across different populations (children, adolescents and adults, left--handers and individuals with developmental dyslexia) that supports the claims that hemispheric lateralization is graded rather than binary and that this graded organization emerges dynamically over the course of development. Perturbations of this system either during development or in adulthood provide further insights into the principles governing hemispheric organization.