As with many other musical traits, the social environment is a key influence on the development of singing ability. While the familial singing environment is likely to be formative, its role relative to other environmental influences such as training is unclear. We used structural equation modeling to test relationships among demographic characteristics, familial environmental variables (early and current singing with family), vocal training, and singing ability in a large, previously documented sample of Australian twins (N = 1163). Notably, early singing with family, and to a lesser extent vocal training, predicted singing ability, whereas current singing with family did not. Early familial singing also mediated the relationship between sex and singing ability, with men who sang less with family during childhood showing poorer ability. Bivariate twin models between early familial singing and singing ability showed the phenotypic correlation was largely explained by shared environmental influences. This raises the possibility of a sensitive period for singing ability, with sociocultural expectations around singing potentially differentiating the developmental trajectories of this skill for men and women.