Variability, Validity and Operator Reliability of Three Ultrasound Systems for Measuring Tissue Stiffness: A Phantom Study

Abstract

Introduction Ultrasound elastography is a method of measuring soft tissue stiffness to detect the presence of pathology. There are several ultrasound elastography devices on the market. The aim of this study was twofold. Firstly, to determine the validity of three different ultrasound systems used to measure tissue stiffness. Secondly, to determine the operator reliability and repeatability when using these three systems. Materials and methods Two observers undertook multiple stiffness measurements from a phantom model using three different ultrasound systems; the LOGIQ E9, the Aixplorer, and the Acuson S2000. The phantom model had four cylindrical-shaped inclusions (Type 1-4) of increasing stiffness values and diameter embedded within. The background phantom stiffness was fixed. The mean, standard deviation, and coefficient of variation (CV) were calculated from measured stiffness readings per diameter per inclusion. Intra-observer variability was assessed. The validity of the measured stiffness value was assessed by calculating the difference between the measured elasticities and actual phantom elasticities. Bland-Altman plots with limits of agreement were used to display the inter-observer agreement. The intraclass correlation coefficients (ICC) were used to measure intra-observer, inter-observer, and inter-system reliability. Results Each observer undertook 1020 measurements. All three systems generally underestimated the stiffness values for the inclusions; the higher the actual stiffness value, the more significant the underestimation. The percentage difference between measured stiffness and actual stiffness varied from -79.1% to 12.7%. The intra-observer variability was generally less than 5% for observers using the LOGIQ E9 and the Aixplorer systems but more than 10% over the stiffer inclusions (Types 3 and 4) for the Acuson system. There was 'almost perfect' intra-observer reliability and repeatability for both the LOGIQ E9 and the Aixplorer systems; this was 'moderate' for the Acuson system over specific inclusions. For all systems, there was 'almost perfect' inter-observer reliability and repeatability between Observer A and Observer B. The inter-system reliability and repeatability were 'almost perfect' between the LOGIQ E9 system and the Aixplorer system but 'poor' and 'moderate' when the Acuson system was matched with the LOGIQ E9 system and the Aixplorer system, respectively. Conclusion This study has demonstrated that the Acuson, LOGIQ E9, and Aixplorer ultrasound systems have low variability, high reproducibility, and good intra-observer and inter-observer reliability when used to measure tissue stiffness. However, they all underestimated the stiffness values during this in vitro study. This study also revealed that not all ultrasound systems are comparable when measuring tissue stiffness, with some having better inter-system reliability than others. Ongoing standardization of technology is required at the manufact...

Quantifying Mechanical Properties of the Patella and Achilles Tendon Using Ultrasound Shear Wave Elastography

“…A study by Alrashed et al found measurement differences between systems of up to 15% for superficial structures and 38.6%-82.9% for deeper [16]. This sentiment was reinforced by Javed et al calling for better standardization of systems [17].…”

Is musculoskeletal pain associated with increased muscle stiffness? Evidence map and critical appraisal of muscle measurements using shear wave elastography

“…The reliability of SWE measurements differs across muscles (Davis et al, 2019) and may depend on the size and positioning of the region of interest (Alfuraih et al, 2018). Inter‐system reliability is not granted (Javed et al, 2022; Long et al, 2018; Mulabecirovic et al, 2016). The reliability of SWE measurements should be documented for the conditions of the study (Alfuraih et al, 2017).…”

“…Expert reviews indicate open questions regarding the interpretation of SWE images (Davis et al, 2019; Ferraioli et al, 2022). Based on methodological literature (Davis et al, 2019; Ferraioli et al, 2022; Javed et al, 2022; Lin et al, 2017; Rominger et al, 2018) and the authors' experience, 11 supplementary items were developed to specify aspects relevant to Items 9 and 11 of the AXIS tool (parameters' measurement and description of methods, respectively) and to compare the SWE methods between studies. The supplementary items targeted the replicability of a study by sufficiently reported information, the control of the image and the measurement quality, the reliability and the representativeness of the measurements, data processing and the statistical analysis.…”