Objectives To identify factors that were predictive of improved pain status among older adults with chronic back pain participating in the Adaptive Physical Activity (APA) program and to identify factors that were predictive of adherence to APA. Methods An observational cohort study of 392 older adults (ages 50–88) with chronic back pain participating in APA for 12 months. APA was a community-based group exercise program given for 1-hour, twice weekly, in local gyms. Primary outcome measures were improved pain based on a global rating of change evaluation and adherence to the APA program (defined as participation in >75% of exercise sessions). Potential predictor variables were entered into multivariate logistic regression models to determine the most accurate set of variables for predicting improved pain and adherence. Results Presence of depressive symptoms, poor self-rated health and adherence to APA were the best predictors of improved pain status, with adherence being the strongest predictor [Odds Ratio:13.88 (95% CI: 8.17, 23.59)] Better physical function, longer pain duration and positive rating of the trainer were all positively associated with adherence to APA; whereas poor self-rated health and further distance from the gym were inversely associated. Conclusions Given that adherence to APA is the key predictor of improved back pain, future efforts should focus on strategies to improve adherence. Our data suggest that enhanced training of exercise trainers, development of separate classes for people with different functional levels and use of psychosocial interventions to reduce health pessimism and depression may be potential targets for improving adherence.