Reflections of the real‐world in the unreal, using simulation to design complex real‐world validation studies for spectroscopy

Abstract

Increasing use of real-world experiments embeds scientific study in settings relevant to everyday life but brings a number of complications. Traditional study design has evaluated experimental risk based on a limited number of factors that are tightly controlled while others are standardised. In real-world data, many unstandardised factors risk confounding the study in unanticipated ways. Some simulation tools and processes are evaluated for use in capturing current knowledge of complex scenarios to evaluate validation of multivariate models in real-world settings. Further, it examines the value of probing the range of uncertainty of the assumptions used to build the model. Copulas simulate correlation of multiple underlying factors where the web of causal interactions cannot be untangled. These factors influence the biochemical status of an individual, determining what pathways are activated in that individual. The final biochemical status of the individuals is then transformed into Raman spectroscopic, reference and pathological measurements. Evaluating different validation scenarios (same population, subset of that population and a new population) allows evaluation of the risks of the model being deployed for different intended uses. Technical factors are stress tested to determine the analytical bottlenecks. Individual socio-economic factors are stress tested todetermine the leverage that current uncertainty has on the risks to the validation study. Use of simulation was able to identify the limitations of the current model for new scenarios and also to assess the specific changes in expected sensitivity versus specificity in the new populations, which would be relevant for benefit risk analysis.

Tribute to Derek Long: An instant snapshot of the development of Raman spectroscopy and its application in the fields of instrumentation and methodology, solid‐state materials, cultural heritage, DFT modeling and applications in biology, microbiology, and medicine

“…Some regulatory authorities have started to more precisely define what real-world data and evidence is, with the FDA effectively stating that it is any data that would have been collected anyway as a matter of routine care ("Real-World Data (RWD) are data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources"). Renwick Beattie (Reflections of the real-world in the unreal, using simulation to design complex realworld validation studies spectroscopy, Queen's University, UK) [56] proposes reflections of the real-world in the unreal, using simulation to design complex realworld validation studies for spectroscopy. Key advantages of real-world data are the relative low cost of obtaining data and the increased relevance of the results to the actual world around us.…”