Toward a Framework for Outcome-Based Analytical Performance Specifications: A Methodology Review of Indirect Methods for Evaluating the Impact of Measurement Uncertainty on Clinical Outcomes

Smith, Alison; Shinkins, Bethany; Hall, Peter; Hulme, Claire; Messenger, Mike

doi:10.1373/clinchem.2018.300954

Cited by 28 publications

(25 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The purpose of this program is to explore the relation between diagnostic accuracy measures and measurement uncertainty, as diagnostic accuracy is fundamental to clinical decision making, while defining the permissible measurement uncertainty is critical to quality and risk management in laboratory medicine. There has been extensive research on either diagnostic accuracy or measurement uncertainty, however very limited on both subjects (14,29,30).…”

Section: Discussionmentioning

confidence: 99%

“…Although the estimation of measurement uncertainty is essential for quality assurance in laboratory medicine (11), its effect on clinical decision making and consequently on clinical outcomes is rarely quantified (14). As direct outcome studies are very complex, a feasible first step is exploring the effect of measurement uncertainty on misclassification (15) and subsequently on diagnostic accuracy measures and the corresponding risk.…”

Section: Uncertaintymentioning

confidence: 99%

See 1 more Smart Citation

WITHDRAWN: A software tool for exploring the relation between diagnostic accuracy and measurement uncertainty

Chatzimichail¹,

Hatjimihail²

2020

Preprint

View full text Add to dashboard Cite

Background Screening and diagnostic tests are used to classify people with and without a disease. Diagnostic accuracy measures are used to evaluate the correctness of a classification in clinical research and practice. Although the correctness of a classification based on a measurand depends on the uncertainty of measurement, there has been limited research on their relation. The objective for this work is to develop an exploratory tool for the relation between diagnostic accuracy measures and measurement uncertainty, as diagnostic accuracy is fundamental to clinical decision making, while measurement uncertainty is critical to quality and risk management in laboratory medicine. Results For this reason, a freely available interactive program has been developed, written in Wolfram Language. The program provides four modules for calculating, optimizing, plotting and comparing various diagnostic accuracy measures and the corresponding risk of diagnostic or screening tests measuring a normally distributed measurand, applied at a single point in time in non-diseased and diseased populations. This is done for differing prevalence of the disease, mean and standard deviation of the measurand, diagnostic threshold, standard measurement uncertainty of the tests and expected loss. The application of the program is illustrated with a case study of glucose measurements in diabetic and non-diabetic populations, that demonstrates the relation between diagnostic accuracy measures and measurement uncertainty. Conclusion The presented interactive program is user-friendly and can be used as a flexible educational and research tool in medical decision making, to explore the relation between diagnostic accuracy measures and measurement uncertainty.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Uncertaintymentioning

confidence: 99%

WITHDRAWN: A software tool for exploring the relation between diagnostic accuracy and measurement uncertainty

Chatzimichail¹,

Hatjimihail²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…There has been extensive research on either diagnostic accuracy or measurement uncertainty; however, such research is very limited on both subjects [ 14 , 38 , 39 ].…”

Section: Discussionmentioning

confidence: 99%

“…Although the estimation of measurement uncertainty is essential for quality assurance in laboratory medicine [ 11 ], its effect on clinical decision-making and consequently on clinical outcomes is rarely quantified [ 14 ]. As direct-outcome studies are very complex, a feasible first step is exploring the effect of measurement uncertainty on misclassification [ 15 ] and subsequently on diagnostic accuracy measures and the corresponding risk.…”

Section: Introductionmentioning

confidence: 99%

A Software Tool for Exploring the Relation between Diagnostic Accuracy and Measurement Uncertainty

Chatzimichail¹,

Hatjimihail²

2020

Diagnostics

View full text Add to dashboard Cite

Screening and diagnostic tests are used to classify people with and without a disease. Diagnostic accuracy measures are used to evaluate the correctness of a classification in clinical research and practice. Although this depends on the uncertainty of measurement, there has been limited research on their relation. The objective of this work was to develop an exploratory tool for the relation between diagnostic accuracy measures and measurement uncertainty, as diagnostic accuracy is fundamental to clinical decision-making, while measurement uncertainty is critical to quality and risk management in laboratory medicine. For this reason, a freely available interactive program was developed for calculating, optimizing, plotting and comparing various diagnostic accuracy measures and the corresponding risk of diagnostic or screening tests measuring a normally distributed measurand, applied at a single point in time in non-diseased and diseased populations. This is done for differing prevalence of the disease, mean and standard deviation of the measurand, diagnostic threshold, standard measurement uncertainty of the tests and expected loss. The application of the program is illustrated with a case study of glucose measurements in diabetic and non-diabetic populations. The program is user-friendly and can be used as an educational and research tool in medical decision-making.

show abstract

“…Laboratory results play a fundamental role in clinical decision-making, so the analytical quality must be guaranteed for proper treatment and the error rates must be controlled for patients' safety. 1,2 The examination results are influenced by the in vitro diagnosis detecting systems comprehensively, which consist of analyzers, corresponding assay reagents, calibrators and relevant clinical laboratories, etc Matched or validated analytical systems are the preferred choices for laboratories, where commercial kit methods are ready to be apply on a dedicated instrument. 3 The manufacturer/ method developer verifies and claims the performance characteristics of the validated systems or examination procedure, which only need to be simply validated by the laboratories.…”

Section: Introductionmentioning

confidence: 99%

Sigma metrics application for validated and non‐validated detecting systems performance assessment

Xia

et al. 2020

Clinical Laboratory Analysis

View full text Add to dashboard Cite

Background: Sigma metrics provide an objective and quantitative methodology for analytical quality evaluation of clinical laboratory. This study investigated the testing performance of validated systems and non-validated systems based on sigma metrics, and explored the major parameters affecting the system performance. Methods: Sigma metrics were evaluated by six biochemistry assays based on Beckman and Mindray validated and non-validated systems through crossing the reagents and analyzers. Imprecision and bias were assessed for all assays based on trueness programs organized by National Centre for Clinical Laboratory. Total error allowance obtained from the Chinese Ministry of Health Clinical Laboratory Centre Industry Standard (WS/T403-2012). Results: The imprecision for all systems meets the quality specifications except TP assay (2.19%) detected by Mindray non-validated system, and the bias for four assays measured by non-validated systems cannot fulfill the criterion, including lactate dehydrogenase (LDH), total protein (TP), triglycerides (TG), and glucose (GLU). Higher biases were detected in six assays at different levels among non-validated and validated systems. Systems performed poorly or unacceptably for TP assay with sigma metrics lower than 3 except Mindray non-validated system. The sigma metrics for other assays with four systems were greater than 3 except the LDH evaluated on Mindray non-validated systems. Conclusion: Non-validated systems may introduce performance uncertainty compared with validated systems based on sigma metrics evaluation, and lower bias was provided by validated systems. The performance of non-validated systems should be evaluated thoroughly in the clinical laboratory before they were adopted for routine use.

show abstract

Toward a Framework for Outcome-Based Analytical Performance Specifications: A Methodology Review of Indirect Methods for Evaluating the Impact of Measurement Uncertainty on Clinical Outcomes

Cited by 28 publications

References 89 publications

WITHDRAWN: A software tool for exploring the relation between diagnostic accuracy and measurement uncertainty

WITHDRAWN: A software tool for exploring the relation between diagnostic accuracy and measurement uncertainty

A Software Tool for Exploring the Relation between Diagnostic Accuracy and Measurement Uncertainty

Sigma metrics application for validated and non‐validated detecting systems performance assessment

Contact Info

Product

Resources

About