No-gold-standard evaluation of image-acquisition methods using patient data

Abstract

Several new and improved modalities, scanners, and protocols, together referred to as image-acquisition methods (IAMs), are being developed to provide reliable quantitative imaging. Objective evaluation of these IAMs on the clinically relevant quantitative tasks is highly desirable. Such evaluation is most reliable and clinically decisive when performed with patient data, but that requires the availability of a gold standard, which is often rare. While no-gold-standard (NGS) techniques have been developed to clinically evaluate quantitative imaging methods, these techniques require that each of the patients be scanned using all the IAMs, which is expensive, time consuming, and could lead to increased radiation dose. A more clinically practical scenario is where different set of patients are scanned using different IAMs. We have developed an NGS technique that uses patient data where different patient sets are imaged using different IAMs to compare the different IAMs. The technique posits a linear relationship, characterized by a slope, bias, and noise standard-deviation term, between the true and measured quantitative values. Under the assumption that the true quantitative values have been sampled from a unimodal distribution, a maximum-likelihood procedure was developed that estimates these linear relationship parameters for the different IAMs. Figures of merit can be estimated using these linear relationship parameters to evaluate the IAMs on the basis of accuracy, precision, and overall reliability. The proposed technique has several potential applications such as in protocol optimization, quantifying difference in system performance, and system harmonization using patient data.

Development of the quantitative PET prostate phantom (Q3P) for improved quality assurance of 18F‐PSMA PET imaging in metastatic prostate cancer

“…These two parts had matching rabbets for alignment, and the parts were solvent welded together. We designed 13 cylindrical inserts in total, to create fillable spherical chambers of the following sizes: 3, 5, 7, 10, 13,15,17,22,25,28,30,33,37 (3, 5, 7, 10, 13, 17 mm) are shown in Figure 2d. Each cylinder has a 3.18 mm (1/8″) hole for 50 mm with a 1/4-28 thread at the proximal end.…”

“…Image processing methods may be compared using clinical trials, but they are time-consuming and suffer from limited sample sizes, resources, and unreliable ground truth. 13,14 Phantoms are particularly useful for evaluating new imaging techniques, as they provide a controlled environment that mimics the properties of a human subject without posing a risk to patients. For instance, phantoms are commonly used to obtain standard measurements of spatial resolution, sensitivity, and contrast recovery.…”

Development of scalable lymphatic system in the 4D XCAT phantom: Application to quantitative evaluation of lymphoma PET segmentations

“…4 Delineating the ground truth for human subjects, such as the location or volume of a lesion, is fundamentally a time-consuming and challenging task to perform within research studies. 4,5 Phantoms are routinely used in imaging research and clinical practice to assess image quality and quantitative accuracy of images. [6][7][8][9][10] However, few phantoms emulate the anatomical structures and heterogeneity features that are present in human subjects with a high degree of realism.…”

“…The implementation of these methods continues to exceed the pace by which these techniques can be properly validated and optimized. Clinical trials are slow to implement and often suffer from limited sample sizes, resources, and funding 4 . Delineating the ground truth for human subjects, such as the location or volume of a lesion, is fundamentally a time‐consuming and challenging task to perform within research studies 4,5 .…”

“…Clinical trials are slow to implement and often suffer from limited sample sizes, resources, and funding 4 . Delineating the ground truth for human subjects, such as the location or volume of a lesion, is fundamentally a time‐consuming and challenging task to perform within research studies 4,5 . Phantoms are routinely used in imaging research and clinical practice to assess image quality and quantitative accuracy of images 6–10 .…”