In mammals, birth mass is an important predictor of early growth and survival. Within litters, heavier siblings are usually able to outcompete smaller siblings and gain more resources, thereby often permanently shaping phenotypic development. Early body size and growth are particularly important for later fitness. Only few studies investigated if and how differences within the early family environment contribute to long-term variation in fitness among individuals. We quantified if initial differences in size translate to size differences in adulthood and whether birth mass, relative size within the litter, litter size or the litter sex-ratio affect maturation and reproductive output of female wild cavies (Cavia aperea). Initial differences in mass were maintained until animals reached maximum adult mass at two years of age. Heavier sisters matured earlier and invested more into their first litter than smaller sisters, presumably because smaller sisters invested more into their own growth during the first pregnancy. Growing up in mixed-sexed litters in comparison to female-only litters slowed down maturation in smaller but not the heaviest female within a litter and had no effect on female reproductive effort. Variation in reproduction of multiparous females was to a lesser extent explained by the initial relative size of siblings. Offspring survival to independence was high but slightly lower when mothers had been born as smaller sisters. Our results demonstrate that factors of the early family environment not only affect immediate offspring development but lead to long-term fitness consequences.