Body concerns and stress-related disorders are increasing in the younger population in a wide range of nations. Studies find links between both self-worth, exercise dependence, and self-esteem in relation to stress, but few have considered all three variables in relation to one another. The present study explored whether the co-appearance of high levels of psychological distress, and low levels of self-esteem may be a vulnerability factor for developing exercise dependence by studying the links between self-esteem, psychological stress, and exercise dependence. A standardized cross-sectional questionnaire was completed by 203 regular exercisers attending two gyms (mean age: 35.9 years). The variables self-esteem, psychological distress, and exercise dependence were all significantly correlated with each other, even after weekly exercise amount, age, and gender had been accounted for. Those who exercised for more than 9 h per week had a significantly higher score on stress and exercise dependence symptoms, and a lower score on self-esteem compared with the remaining groups. One could hypothesize that low self-esteem is a vulnerability factor and high psychological stress a maintenance factor for an exercise-dependent person. It is argued that more focus should be directed toward the negative consequences of excessive exercise.