Methodology of method comparison studies evaluating the validity of cardiac output monitors: a stepwise approach and checklist † †This Article is accompanied by Editorial Aew110.

Abstract

The validity of each new cardiac output (CO) monitor should be established before implementation in clinical practice. For this purpose, method comparison studies investigate the accuracy and precision against a reference technique. With the emergence of continuous CO monitors, the ability to detect changes in CO, in addition to its absolute value, has gained interest. Therefore, method comparison studies increasingly include assessment of trending ability in the data analysis. A number of methodological challenges arise in method comparison research with respect to the application of Bland-Altman and trending analysis. Failure to face these methodological challenges will lead to misinterpretation and erroneous conclusions. We therefore review the basic principles and pitfalls of Bland-Altman analysis in method comparison studies concerning new CO monitors. In addition, the concept of clinical concordance is introduced to evaluate trending ability from a clinical perspective. The primary scope of this review is to provide a complete overview of the pitfalls in CO method comparison research, whereas other publications focused on a single aspect of the study design or data analysis. This leads to a stepwise approach and checklist for a complete data analysis and data representation.

Accuracy and precision of transcutaneous carbon dioxide monitoring: a systematic review and meta-analysis

“…Therefore, methods other than the Bland-Altman approach may be more suited to quantitatively assess trending ability. For example, the 4Q, polar analysis and clinical concordance methods have recently been recommended for consideration in method comparison studies evaluating the validity of cardiac output monitors 21. Similar to TcCO 2 monitoring, cardiac output monitors provide a continuous estimate of a dynamic physiological parameter, which changes rapidly from various influences.…”

The haemodynamic dilemma in emergency care: Is fluid responsiveness the answer? A systematic review

“…More recently this range has narrowed with a 10–15% change commonly used to define a change that is not due to random variability [49]. This cut-off is sometimes derived from the reported precision of thermodilution [50], and may not be optimal when a different device is used [51]. The clinical significance of a given change should be also appreciated.…”

Accuracy and precision of non-invasive cardiac output monitoring by electrical cardiometry: a systematic review and meta-analysis

“…Despite its inability to measure absolute CO values, which is assessed by the Bland-Altman analysis, EC could still be applicable as trend monitor. To achieve acceptable trending ability, good precision is required, independent of the accuracy [91]. For the assessment of trending ability different methods are described, of which the for-quadrant plot and the polar plot are recommended [92][93][94].…”

“…The ICON ® /Aesculon ® monitor is a device in development and future research should clarify its place between existing hemodynamic monitoring devices. The high risk of bias in the statistical analysis domain of the modified QUADAS-2 tool emphasizes the lack of consensus how to present data in validation studies, despite the fact that good proposals have been published [20,25,87,91]. Consensus is required to interpret results of different studies and draw conclusions.…”