Background: Environmental variance (V E) is partly under genetic control and has recently been proposed as a measure of resilience. Unravelling the genetic background of the V E of complex traits could help to improve resilience of livestock and stabilize their production across farming systems. The objective of this study was to identify genes and functional mutations associated with variation in V E of litter size (LS) in rabbits. To achieve this, we combined the results of a genome-wide association study (GWAS) and a whole-genome sequencing (WGS) analysis using data from two divergently selected rabbit lines for high and low V E of LS. These lines differ in terms of biomarkers of immune response and mortality. Moreover, rabbits with a lower V E of LS were found to be more resilient to infections than animals with a higher V E of LS. Results: By using two GWAS approaches (single-marker regression and Bayesian multiple-marker regression), we identified four genomic regions associated with V E of LS, on chromosomes 3, 7, 10, and 14. We detected 38 genes in the associated genomic regions and, using WGS, we identified 129 variants in the splicing, UTR, and coding (missense and frameshift effects) regions of 16 of these 38 genes. These genes were related to the immune system, the development of sensory structures, and stress responses. All of these variants (except one) segregated in one of the rabbit lines and were absent (n = 91) or fixed in the other one (n = 37). The fixed variants were in the HDAC9, ITGB8, MIS18A, ENSOCUG00000021276 and URB1 genes. We also identified a 1-bp deletion in the 3′UTR region of the HUNK gene that was fixed in the low V E line and absent in the high V E line. Conclusions: This is the first study that combines GWAS and WGS analyses to study the genetic basis of V E. The new candidate genes and functional mutations identified in this study suggest that the V E of LS is under the control of functions related to the immune system, stress response, and the nervous system. These findings could also explain differences in resilience between rabbits with homogeneous and heterogeneous V E of litter size.