Estimating the complex relationship between fitness and genotype or phenotype (i.e. the adaptive landscape) is one of the central goals of evolutionary biology. Empirical fitness landscapes have now been estimated for numerous systems, from phage to proteins to finches. However, the nature of adaptive walks connecting genotypes to organismal fitness, speciation, and novel ecological niches are still poorly understood. One outstanding system for addressing these connections is a recent adaptive radiation of ecologically and morphologically distinct pupfishes (a generalist, molluscivore, and scale-eater) endemic to San Salvador Island, Bahamas. Here, we leveraged whole-genome sequencing of 139 hybrids from two independent field fitness experiments to identify the genomic basis of fitness, visualize the first genotypic fitness networks in a vertebrate system, and infer the contributions of different sources of genetic variation to the accessibility of the fitness landscape. We identified 132 SNPs that were significantly associated with fitness in field enclosures, including six associated genes that were differentially expressed between specialists, and one gene (protein-lysine methyltransferase: METTL21E) misexpressed in hybrids, suggesting a potential intrinsic genetic incompatibility. We then constructed genotypic fitness networks from adaptive alleles and show that only introgressed and de novo variants, not standing genetic variation, increased the accessibility of genotypic fitness paths from generalist to specialists. Our results suggest that adaptive introgression and de novo variants provided key connections in adaptive walks necessary for crossing fitness valleys and triggering the evolution of novelty during adaptive radiation.