Avoiding a reproducibility crisis in regulatory toxicology—on the fundamental role of ring trials

Abstract

The ongoing transition from chemical hazard and risk assessment based on animal studies to assessment relying mostly on non-animal data, requires a multitude of novel experimental methods, and this means that guidance on the validation and standardisation of test methods intended for international applicability and acceptance, needs to be updated. These so-called new approach methodologies (NAMs) must be applicable to the chemical regulatory domain and provide reliable data which are relevant to hazard and risk assessment. Confidence in and use of NAMs will depend on their reliability and relevance, and both are thoroughly assessed by validation. Validation is, however, a time- and resource-demanding process. As updates on validation guidance are conducted, the valuable components must be kept: Reliable data are and will remain fundamental. In 2016, the scientific community was made aware of the general crisis in scientific reproducibility—validated methods must not fall into this. In this commentary, we emphasize the central importance of ring trials in the validation of experimental methods. Ring trials are sometimes considered to be a major hold-up with little value added to the validation. Here, we clarify that ring trials are indispensable to demonstrate the robustness and reproducibility of a new method. Further, that methods do fail in method transfer and ring trials due to different stumbling blocks, but these provide learnings to ensure the robustness of new methods. At the same time, we identify what it would take to perform ring trials more efficiently, and how ring trials fit into the much-needed update to the guidance on the validation of NAMs.

Addressing chemically-induced obesogenic metabolic disruption: selection of chemicals for in vitro human PPARα, PPARγ transactivation, and adipogenesis test methods

“…Historically, for many OECD in vitro Test Guidelines (TGs), the initial focus has been on (dichotomous) hazard identification particularly for classification and labelling needs, but also based upon the extent to which the results from the validation study can be reliably interpreted. However, to enable in vitro test methods to be used beyond prioritisation for subsequent testing in animal studies, to develop utility within IATAs, it is of value if a test method can also provide continuous (quantitative) data to inform upon the potency of a chemical ( 47 , 52 ). This will enable the characterisation of the influence of physico-chemical properties on variability across the concentration response being generated ( 52 ).…”

“…However, to enable in vitro test methods to be used beyond prioritisation for subsequent testing in animal studies, to develop utility within IATAs, it is of value if a test method can also provide continuous (quantitative) data to inform upon the potency of a chemical ( 47 , 52 ). This will enable the characterisation of the influence of physico-chemical properties on variability across the concentration response being generated ( 52 ).…”