West Nile (WN) virus infection of humans is frequently asymptomatic, but can also lead to WN fever or neuroinvasive disease. CD4 T cells and B cells are critical in the defense against WN virus, and neutralizing antibodies, which are directed against the viral glycoprotein E, are an accepted correlate of protection. For the efficient production of these antibodies, B cells interact directly with CD4 helper T cells that recognize peptides from E or the two other structural proteins (capsid-C and membrane-prM/M) of the virus. However, the specific protein sites yielding such helper epitopes remain unknown. Here, we explored the CD4 T cell response in humans after WN virus infection using a comprehensive library of overlapping peptides covering all three structural proteins. By measuring T cell responses in 29 individuals with either WN virus disease or asymptomatic infection, we showed that CD4 T cells focus on peptides in specific structural elements of C and at the exposed surface of the pre-and postfusion forms of the E protein. Our data indicate that these immunodominant epitopes are recognized in the context of multiple different HLA molecules. Furthermore, we observed that immunodominant antigen regions are structurally conserved and similarly targeted in other mosquito-borne flaviviruses, including dengue, yellow fever, and Zika viruses. Together, these findings indicate a strong impact of virion protein structure on epitope selection and antigenicity, which is an important issue to consider in future vaccine design.