Our previous research demonstrated that genetic distance (GD) on effective mutation (EM) sites can be used to evaluate vaccine effectiveness (VE) in silico in real time. This study further investigates the relationship between VE and GD on antigenic sites (AS) and identifies key amino acid sites related to vaccine protection against influenza A/H1N1pdm09 and A/H3N2 between 2009 and 2019 flu seasons. We found that not any AS on hemagglutinin (HA) and neuraminidase (NA) may cause a decrease in VE, rather, GD on the intersection set of EM and AS is highly predictive of influenza VE. The integrated GD of HA and NA can explain up to 87.8% of VE variations for H3N2. Significant improvement is also found for VE prediction for pH1N1. Accurate prediction of influenza VE before vaccine deployment may facilitate reverse vaccinology to optimize vaccine antigen design and facilitate government preparedness of epidemics.