Inbreeding depression, the loss of offspring fitness due to consanguineous mating, is generally detrimental for individual performance and population viability. We therefore investigated inbreeding effects in a declining population of Antarctic fur seals at Bird Island, South Georgia. Here, localised warming has reduced the availability of the seal's staple diet, Antarctic krill, leading to a temporal increase in the strength of viability selection against inbred offspring, which are increasingly failing to recruit into the adult breeding population. However, it remains unclear whether viability selection operates before or after nutritional independence at weaning. We therefore used microsatellite data from 884 pups and their mothers, and SNP array data from 100 mother-offspring pairs, to quantify the effects of individual and maternal inbreeding on three important neonatal fitness traits: birth mass, survival and growth. We did not find any clear or consistent effects of inbreeding on any of these traits. This suggests that viability selection filters inbred individuals out of the population as juveniles during the time window between weaning and recruitment. Our study brings into focus a poorly understood life-history stage and emphasises the importance of understanding the ecology and threats facing juvenile pinnipeds.