Mpox is an infectious disease caused by the monkeypox virus (MPXV) belonging to the Orthopoxvirus (OPXV) genus, which includes smallpox and vaccinia virus (VACV). A global mpox outbreak which began in May 2022 has infected more than 88,000 people. VACV-based vaccines provide protection against mpox disease but complicate the use of serological assays for disease surveillance. We tested the reactivity of serum IgG from Modified Vaccinia Ankara-Bavarian Nordic (MVA-BN)-vaccinated (n = 12) and convalescent mpox-infected (n = 5) individuals and uninfected, non-vaccinated controls (n = 32) to MPXV/VACV proteins A27, A29, A30, A35, B16, B21, C19, D6, E8, H3, I1, and L1. Using a subset of MPXV antigen-based assays (A35, B16, E8, H3, and I1), we conducted a mpox antibody survey of serum from 214 individuals, including 117 (54.7%) people with HIV (PWH) collected between June 2022 and January 2023, excluding individuals who reported recent mpox vaccination or infection, and 32 young, pre-pandemic controls. The convalescent sera reacted strongly to most tested antigens. Vaccine sera responses were limited to A35, E8, H3, and I1. IgG antibody to E8 was markedly elevated in all vaccinated individuals. B16 IgG showed high sensitivity (100% [95% CI: 56.55–100.0%]) and specificity (91.67% [64.61–99.57%]) for distinguishing infection from MVA-BN vaccination, while E8 IgG showed 100% [75.75–100] sensitivity and 100% [79.61–100] specificity for detecting and distinguishing vaccinated individuals from controls. We identified 11/214 (5.1%) recent serum samples and 1/32 (3.1%) young, pre-pandemic controls that were seropositive for ≥2 MPXV antibodies, including 6.8% of PWH. Seropositivity was 10/129 (7.8%) among males compared to 1/85 (1.2%) among females. Our findings provide insight into the humoral immune response to mpox and demonstrate the usefulness of inexpensive, antigen-based serosurveillance in identifying asymptomatic or unreported infections.