Validating scales for clinical purposes is a common process in medicine and psychology. Using machine-learning and statistics, we revalidated the Fenigstein & Vanable Paranoia Scale and showed that these kinds of approaches could be used both to achieve construct validity and criterion validity and could thus add an additional layer of evidence to traditional validation approaches. However, there is still a lot of work needed in order to evaluate the whole range of applications, disadvantages, and potential limitations of these approaches when applied for psychometric purposes.

Validating scales for clinical use is a common procedure in medicine and psychology. Through the application of computational methods, we present a new strategy for estimating construct validity and criterion validity. XGBoost, Random Forest and Support-Vector machine learning algorithms were employed in order to make predictions based on the pattern of participants’ responses by systematically controlling computational experiments with artificial experiments whose results are guaranteed. According to these findings, these approaches are capable of achieving construct and criterion validity and therefore could provide an additional layer of evidence to traditional validation approaches. In particular, this study examined the extent to which measured items are inferable by theoretically related items, as well as the extent to which the information carried by a given construct can be translated into other theoretically compatible normative scales based on other constructs (thereby providing information about construct validity); as well as the replicability of clinical decision rules on several partitions (thereby providing information about criterion validity).

No abstract