Viral infections have variable outcomes with severe disease occurring in only few individuals. We hypothesized that this variable outcome could correlate with the nature of responses made to previous microbes. To test this, mice were infected initially with IAV and in memory-phase challenged with LCMV, which we show here to have relatively minor cross-reactivity with IAV. The outcome in genetically identical mice varied from mild pneumonitis to severe acute lung injury with extensive pneumonia and bronchiolization, similar to that observed in patients that died of the 1918 H1N1 pandemic. Lesion expression did not correlate with virus titers. Instead, disease severity directly correlated with and was predicted by the frequency of IAV-PB1703- and -PA224-specific responses, which crossreacted with LCMV-GP34 and -GP276, respectively. Eradication or functional ablation of these pathogenic memory T-cell populations, using mutant-viral strains, peptide-based tolerization strategies, or short-term anti-IFNγ treatment inhibited severe lesions such as bronchiolization from occurring. Heterologous immunity can shape outcome of infections and likely individual responses to vaccination, and can be manipulated to treat or prevent severe pathology.