Global concern over COVID-19 vaccine distribution disparities highlights the need for strategic booster shots. We explored longitudinal antibody responses post-booster during the Omicron wave in a Japanese cohort, emphasizing prior infection and booster doses. This prospective cohort study included 1763 participants aged 18 years and older with at least three vaccine doses (7376 datapoints). Antibody levels were measured every 2 months. We modeled temporal declines in antibody levels after COVID-19 vaccine boosters according to prior infection status and booster doses using a Bayesian linear mixed-effects interval-censored model, considering age, sex, underlying conditions, and lifestyle. Prior infection enhanced post-booster immunity (posterior median 0.346, 95% credible interval [CrI] 0.335–0.355), maintaining antibody levels (posterior median 0.021; 95% CrI 0.019–0.023) over 1 year, in contrast to uninfected individuals whose levels had waned by 8 months post-vaccination. Each additional booster was correlated with higher baseline antibody levels and slower declines, comparing after the third dose. Female sex, older age, immunosuppressive status, and smoking history were associated with lower baseline post-vaccination antibodies, but not associated with decline rates except for older age in the main model. Prior infection status and tailored, efficient, personalized booster strategies are crucial, considering sex, age, health conditions, and lifestyle.