Background Assessing conditioning pain modulation (CPM) with spinal reflex measures may produce more objective and stable CPM effects than using psychophysical measures. The aim of the study was to compare the CPM effect and test–retest reliability between a psychophysical protocol with thermal test‐stimulus and a spinal reflex protocol with electrical test‐stimulus. Methods Twenty‐five healthy volunteers participated in two identical experiments separated by minimum 1 week. The thermal test‐stimulus was a constant heat stimulation of 120 s on the subjects’ forearm with continuous ratings of pain intensity on a 10 cm visual analogue scale. The electrical test‐stimulus was repeated electrical stimulation on the arch of the foot for 120 s, which elicited a nociceptive withdrawal reflex recorded from the anterior tibial muscle. Conditioning stimulus was a 7°C water bath. Differences in the magnitude and test–retest reliability were investigated with repeated‐measures analysis of variance and by relative and absolute reliability indices. Results The CPM effect was −46% and 4.5% during the thermal and electrical test‐stimulus (p < 0.001) respectively. Intraclass correlation coefficient of 0.5 and 0.4 was found with the electrical and thermal test‐stimulus respectively. Wide limits of agreement were found for both the electrical (−3.4 to 3.8 mA) and the thermal test‐stimulus (−3.2 to 3.6 cm). Conclusions More pronounced CPM effect was demonstrated when using a psychophysical protocol with thermal test‐stimulus compared to a spinal reflex protocol with electrical test‐stimulus. Fair relative reliability and poor absolute reliability (due to high intraindividual variability) was found in both protocols. Significance The large difference in CPM effect between the two protocols suggests that the CPM effect relates to pain perception rather than nociception on the spinal level. Due to poor absolute intrarater reliability, we recommend caution and further research before using any of the investigated CPM protocols in clinical decision making on an individual level.