Human anelloviruses (AVs) are extremely genetically diverse, widespread in the human population, and cause chronic infections. However, the evolutionary dynamics of AVs within single hosts is currently unknown, and it is unclear whether these changes have an implication on the long-term persistence of AVs in the host. Here, we assessed the evolutionary dynamics of six AV lineages during 30 years of chronic infection at single host resolution. The total number of substitutions and the number of variable sites increased over time. However, not all substitutions reached population fixation, showing that AV lineages form heterogeneous swarms within the host. Most substitutions occurred within a hypervariable region (HVR) located between nucleotide positions 800 and 1300 of ORF1, which is known to be located within the spike domain. Different regions of the ORF1 gene undergo either positive or negative selection pressure. Sites under strong diversifying selection pressure were detected in the HVR, while the majority of the sites under purifying selection were detected outside this region. The HVR may play the role of an immunological decoy that prevents antibodies from binding to more vulnerable parts of ORF1. Moreover, the frequent substitutions in this region may increase the chances of AV particles escaping immune recognition.