Background: Long-term musculoskeletal pain is a common problem in primary health care settings that is difficult to treat. Two common treatments are mental relaxation and massage. Scientific studies show contradictory results. Furthermore, many studies lack long-term follow-up even though it is a chronic disorder. The purpose of this randomized clinical trial was to assess possible effects of massage as compared to listening to relaxation tapes in conditions of ‘diffuse’ and long-term musculoskeletal pain. Methods: 129 patients from primary health care suffering from long-term musculoskeletal pain were randomized to either a massage or mental relaxation group, and assessed before, during and after treatment. Results: During treatment there was a significant improvement in the three main outcome measures: self-rated health, mental energy, and muscle pain only in the massage group as compared to the relaxation group. However, at the 3-month post-treatment follow-up, there was a significant worsening in the outcome measures (time × group effect p < 0.05) back to initial rating levels in the massage group as compared to no changes in the relaxation group. Conclusion: Massage, but not mental relaxation, is beneficial in attenuating diffuse musculoskeletal symptoms. Beneficial effects were registered only during treatment. This lack of long-term benefits could be due to the short treatment period or treatments such as these do not address the underlying causes of pain. Future studies of long-term pain should include longer treatment periods and post-treatment follow-up. It might also be worthwhile assessing the long-term benefits from booster treatment after the initial intense treatment period.