Systematic and random measurement errors are the cause of uncertain conformance and non-conformance statements. A wrong conformance statement occurs if the workpiece is accepted and is a reject part (type II error or false-negative) and a wrong non-conformance statement if the workpiece is rejected and is an in-spec part (type I error or false-positive). In order to avoid type I and type II errors, measurement uncertainty must be taken into account in the conformance and non-conformance testing. In practice, some procedures are used to consider measurement errors or the uncertainty that deviate from the state-of-the-art in research and technology. As these methods have become established over many years, they are still widely used despite better theoretical knowledge. The standard ISO 14253-1:2017 specifies a procedure based on probability and measurement uncertainty that is in accordance to the internationally accepted “Guide to the expression of uncertainty in measurement” and its supplements but is often not used due to the complexity of the evaluation of measurement uncertainty. In this contribution we give an overview and comparison of the different existing methods and provide an suggestion for supplementing the standard ISO 14253-1:2017, as Monte Carlo simulations enable a direct probability-based conformance and non-conformance testing even for complex measurement processes.