Recent emergence of SARS-CoV-2 Omicron sublineages BA.2.12.1, BA.2.13, BA.4 and BA.5 all contain L452 mutations and show potential higher transmissibility over BA.2. The new variants' receptor binding and immune evasion capability require immediate investigation, especially on the role of L452 substitutions. Herein, coupled with structural comparisons, we showed that BA.2 sublineages, including BA.2.12.1 and BA.2.13, exhibit increased ACE2-binding affinities compared to BA.1; while BA.4/BA.5 shows the weakest receptor-binding activity due to F486V and R493Q reversion. Importantly, compared to BA.2, BA.2.12.1 and BA.4/BA.5 exhibit stronger neutralization escape from the plasma of 3-dose vaccinees and, most strikingly, from vaccinated BA.1 convalescents. To delineate the underlying evasion mechanism, we determined the escaping mutation profiles, epitope distribution and Omicron sublineage neutralization efficacy of 1640 RBD-directed neutralizing antibodies (NAbs), including 614 isolated from BA.1 convalescents. Interestingly, post-vaccination BA.1 infection mainly recalls wildtype (WT) induced humoral memory and elicits antibodies that neutralize both WT and BA.1. These cross-reactive NAbs are significantly enriched on non-ACE2-competing epitopes; and surprisingly, the majority are undermined by R346 and L452 substitutions, namely R346K (BA.1.1), L452M (BA.2.13), L452Q (BA.2.12.1) and L452R (BA.4/BA.5), suggesting that R346K and L452 mutations appeared under the immune pressure of Omicron convalescents. Nevertheless, BA.1 infection can also induce new clones of BA.1-specific antibodies that potently neutralize BA.1 but do not respond to WT SARS-CoV-2, due to the high susceptibility to N501, N440, K417 and E484. However, these NAbs are largely escaped by BA.2 sublineages and BA.4/BA.5 due to D405N and F486V, exhibiting poor neutralization breadths. As for therapeutic NAbs, LY-CoV1404 (Bebtelovimab) and COV2-2130 (Cilgavimab) can still effectively neutralize BA.2.12.1 and BA.4/BA.5, while the S371F, D405N and R408S mutations carried by BA.2/BA.4/BA.5 sublineages would undermine most broad sarbecovirus NAbs. Together, our results indicate that Omicron can evolve mutations to specifically evade humoral immunity elicited by BA.1 infection. The continuous evolution of Omicron poses great challenges to SARS-CoV-2 herd immunity and suggests that BA.1-derived vaccine boosters may not be ideal for achieving broad-spectrum protection.