Early deafness leads to the reorganization of large-scale brain networks, involving and extending beyond the auditory system. Body-centered visuomotor transformation is impaired after early auditory deprivation, associated with a hyper-crosstalk between the task-critical frontoparietal network (FPN) and the default-mode network (DMN). It remains to be elucidated, how the reorganized functional connectivity between the auditory system, the FPN, and the DMN contributes to the impaired visuomotor transformation after early deafness. In this fMRI study, we asked early deaf participants and hearing controls to judge the spatial location of a visual target, either about the middle-sagittal line of their own body (the egocentric task) or another background object (the allocentric task). The bilateral superior temporal gyrus (STG) in the deaf group exhibited cross-modal reorganization, with generally enhanced neural activity during the visual tasks, compared to hearing controls. Moreover, the STG showed significantly increased functional connectivity with both the FPN and the DMN in the deaf group compared to hearing controls, specifically during the egocentric task. The increased STG-FPN and STG-DMN coupling, however, showed antagonistic effects on the egocentric performance of the deaf participants. The increased STG-FPN connectivity was associated with improved (i.e., a beneficial role) while the increased STG-DMN with deteriorated (i.e., a detrimental role) egocentric performance in the deaf participants. No such effect was observed in hearing controls. Therefore, the auditory cortex is reorganized to functionally resemble the FPN in the deaf brain, representing compensatory neuroplasticity to mitigate the impaired visuomotor transformation after early deafness.