The functional organization of left and right hemispheres is different, and hemispheric asymmetries are thought to underlie variations in brain function across individuals. In this study, we assess how differences between hemispheres are reflected in Asymmetric Functional Connectivity (AFC), which provides a full description of how the brain’s connectivity structure during resting state differs from that of the same brain mirrored over the longitudinal fissure. In addition, we assess how AFC varies across subjects. Data were provided by the Human Connectome Project, including 423 resting state and combined language task fMRI data sets, and the pattern of AFC was established for all subjects. While we could quantify the symmetry of brain connectivity at 95%, significant asymmetries were observed, consisting foremost of: (1) higher correlations between language areas in the left hemisphere than between their right hemisphere homologues. (2) Higher correlations between language homologue areas in the right hemisphere and left default mode network, than between language areas in the left hemisphere and the default mode network in the right hemisphere. The extent to which subjects exhibited this pattern correlated with language lateralization and handedness. Further exploration in intersubject variation in AFC revealed several additional patterns, one involving entire hemispheres, and another correlations with limbic areas. These results show that language is an important, but not only determinant of AFC. The additional patterns of AFC require further research to be linked to specific asymmetric neuronal states or events.Electronic supplementary materialThe online version of this article (10.1007/s00429-017-1604-y) contains supplementary material, which is available to authorized users.