Various test methods are available for assessing the susceptibility of materials to solidification cracking during welding. In the widely used Varestraint test, the crack length is selected as a criterion as a function of the applied bending strain. Unfortunately, the crack length does not characterize the material behavior alone but depends to varying degrees on the individual test parameters used, which makes the interpretation of the results difficult. In addition, the crack length is not comparable under different test conditions. To overcome these disadvantages, we have developed a novel evaluation methodology that decouples the machine influence from the material behavior. The measured crack length is related to the maximum possible value specified by welding speed and deformation time. This relative crack length is calculated numerically, considering the orientation of the cracks. Experiments on two high‐alloy martensitic welding consumables show that, in contrast to the conventional evaluation, a comparison of different welding parameters becomes possible. Furthermore, the strain rate proved to be a suitable crack criterion in agreement with Prokhorov's hot cracking model.