Genetic variation underlying differences in organism performance is subject to natural selection, and organisms with high values of genetically determined phenotypic measures of fitness should perform better than those that do not. Using small scleractinian corals (i.e., ≤40-mm diameter), this principle was tested with 20 yr of census data from St. John, US Virgin Islands. Using growth rate (change in diameter) as a measure of fitness, growth in one year was tested for association with growth and survivorship in the following two years, and this process was repeated over 20 yr using a 3-yr sliding window. Virtually all variation in growth was independent of colony size, and growth among pairs of years was highly variable, with corals that grew fast in one year rarely growing fast in the next 2 yr. While growth in some pairs of years was positively correlated, ≤4% of the growth variance was explained by growth in the preceding 2 yr. Survivorship was related positively to growth in the preceding year, but the association was weak, it did not extend over 3 yr, and was inconsistent over the study. These results demonstrate the importance of the environment in translating phenotypic measures of fitness into future performance, and for small Caribbean corals, they suggest that environmental conditions may preempt genotype in determining short-term success.