Although exercising with others may have extra health benefits compared to exercising alone, few studies have examined the differences. We sought to examine whether the association of regular exercise to subjective health status differs according to whether people exercise alone and/or with others, adjusting for frequency of exercise. The study was based on the Japan Gerontological Evaluation Study (JAGES) Cohort Study data. Participants were 21,684 subjects aged 65 or older. Multivariable logistic regression models were used to examine the association. The adjusted odds ratios (ORs) for poor self-rated health were significantly lower for people who exercised compared to non-exercisers. In analyses restricted to regular exercisers the ORs for poor health were 0.69 (95% confidence intervals: 0.60–0.79) for individuals exercising alone more often than with others, 0.74 (0.64–0.84) for people who were equally likely to exercise alone as with others, 0.57 (0.43–0.75) for individuals exercising with others more frequently than alone, and 0.79 (0.64–0.97) for individuals only exercising with others compared to individuals only exercising alone. Although exercising alone and exercising with others both seem to have health benefits, increased frequency of exercise with others has important health benefits regardless of the total frequency of exercise.