Background: Previous studies shows that elite and high-level athletes possess consistently higher pain tolerance to ischemic and cold pain stimulation compared to recreationally active. However, the data previously obtained within this field is sparse and with low consistency. Purpose: The aim of the present study was to examine the difference in pain perception between elite and high-level endurance athletes (cross country skiers and runners), elite soccer players and non-athletes, as well to explore the impact of psychological factors on pain processing. Methods: Seventy one healthy volunteers (33 females and 38 males) participated in the study. Soccer players (n = 17), cross country skiers (n = 12), and long-distance runners (n = 3) formed the athlete group, with 39 non-athletes as controls. Big-five personality traits, fear of pain and Grit (perseverance and passion for long-term goals) were measured prior to induction of experimental pain. Pain threshold and intensity was induced by a PC-controlled heat thermode and measured by a computerized visual analog scale. Pain tolerance was measured by the cold pressor test (CPT). Results: Elite and high-level athletes had increased pain tolerance, higher heat pain thresholds, and reported lower pain intensity to thermal stimulation. Endurance athletes (cross country skiers and long-distance runners) had better tolerance for cold pain compared to both soccer-players and non-athletes. Furthermore, endurance athletes reported lower pain intensity compared to non-athletes, whereas both endurance athletes and soccer players had higher heat pain thresholds compared to non-athletes. Fear of Pain was the only psychological trait that had an impact on all pain measures. Conclusion: The present findings suggest that sports with long durations of physically intense activity, leveling aerobic capacity, are associated with increased ability to tolerate pain and that the amount of training hours has an impact on this tolerance. However, the small sample size implies that the results from this study should be interpreted with caution.