Young children strongly depend on others, and learning to understand their mental states (referred to as Theory of Mind, ToM) is a key challenge of early cognitive development. Traditionally, ToM is thought to emerge around the age of 4 years. Yet, in non-verbal tasks, preverbal infants already seem to consider others' mental states when predicting their actions. These early non-verbal capacities, however, seem fragile and distinct from later-developing verbal ToM. So far, little is known about the nature of these early capacities and the neural networks supporting them. To identify these networks, we investigated the maturation of nerve fiber connections associated with children's correct non-verbal action prediction and compared them with connections supporting verbal ToM reasoning in 3- to 4-year-old children, that is, before and after their breakthrough in verbal ToM. This revealed a ventral network for non-verbal action prediction versus a dorsal network for verbal ToM. Non-verbal capacities were associated with maturational indices in ventral fiber tracts connecting regions of the salience network, involved in bottom-up social attention processes. In contrast, verbal ToM performance correlated with maturational indices of the arcuate fascicle and cingulum, which dorsally connect regions of the default network, involved in higher-order social cognitive processes including ToM in adults. As non-verbal tasks were linked to connections of the salience network, young children may make use of salient perceptual social cues to predict others' actions, questioning theories of mature ToM before 4 years.