Adolescent and young adult (AYA) cancer survivors (18–39 years at diagnosis) often experience negative body changes such as scars, amputation, and disfigurement. Understanding which factors influence body image among AYA survivors can improve age-specific care in the future. Therefore, we aim to examine the prevalence, and association of a negative body image with sociodemographic, clinical, and psychosocial factors, among AYA cancer survivors (5–20 years after diagnosis). A population-based cross-sectional cohort study was conducted among AYA survivors (5–20 years after diagnosis) registered within the Netherlands Cancer Registry (NCR) (SURVAYA-study). Body image was examined via the EORTC QLQ-C30 and QLQ-SURV100. Multivariable logistic regression models were used. Among 3735 AYA survivors who responded, 14.5% (range: 2.6–44.2%), experienced a negative body image. Specifically, AYAs who are female, have a higher Body Mass Index (BMI) or tumor stage, diagnosed with breast cancer, cancer of the female genitalia, or germ cell tumors, treated with chemotherapy, using more maladaptive coping strategies, feeling sexually unattractive, and having lower scores of health-related Quality of Life (HRQoL), were more likely to experience a negative body image. Raising awareness and integrating supportive care for those who experience a negative body image into standard AYA survivorship care is warranted. Future research could help to identify when and how this support for AYA survivors can be best utilized.