AAPM task group report 273: Recommendations on best practices for AI and machine learning for computer‐aided diagnosis in medical imaging

Abstract

Rapid advances in artificial intelligence (AI) and machine learning, and specifically in deep learning (DL) techniques, have enabled broad application of these methods in health care. The promise of the DL approach has spurred further interest in computer‐aided diagnosis (CAD) development and applications using both “traditional” machine learning methods and newer DL‐based methods. We use the term CAD‐AI to refer to this expanded clinical decision support environment that uses traditional and DL‐based AI methods. Numerous studies have been published to date on the development of machine learning tools for computer‐aided, or AI‐assisted, clinical tasks. However, most of these machine learning models are not ready for clinical deployment. It is of paramount importance to ensure that a clinical decision support tool undergoes proper training and rigorous validation of its generalizability and robustness before adoption for patient care in the clinic. To address these important issues, the American Association of Physicists in Medicine (AAPM) Computer‐Aided Image Analysis Subcommittee (CADSC) is charged, in part, to develop recommendations on practices and standards for the development and performance assessment of computer‐aided decision support systems. The committee has previously published two opinion papers on the evaluation of CAD systems and issues associated with user training and quality assurance of these systems in the clinic. With machine learning techniques continuing to evolve and CAD applications expanding to new stages of the patient care process, the current task group report considers the broader issues common to the development of most, if not all, CAD‐AI applications and their translation from the bench to the clinic. The goal is to bring attention to the proper training and validation of machine learning algorithms that may improve their generalizability and reliability and accelerate the adoption of CAD‐AI systems for clinical decision support.

Deep learning-assisted multiple organ segmentation from whole-body CT images

“…Newly developed neural network architectures, loss functions, and image processing algorithms contributed to the improvement of the performance of image segmentation models. Yet, the number of datasets and their diversity remains the bottleneck for successful implementation of deep learning-based algorithms (22). Most studies conveyed the performance of the developed models on a test set excluded from the training set, thus reaching very high Dice coefficients as reported in few challenges held on multiple organ segmentations (23).…”

Regulatory considerations for medical imaging AI/ML devices in the United States: concepts and challenges

“…14 Similarly, other groups and organizations are working to develop consensus best practices for medical imaging AI/ML. [15][16][17][18] Some of the more wide-ranging efforts specific to medical imaging AI/ML include the FUTURE-AI guiding principles developed by five European AI in Health Imaging projects 15 and the American Association of Physicists in Medicine Task Group Report 273 discussing best practices for medical imaging computer-aided diagnosis. 16 In this review paper, we introduce the reader to the medical device regulatory framework within the United States and provide an overview of common elements included in regulatory submissions that incorporate AI/ML models in medical imaging in Sec.…”

A new architecture combining convolutional and transformer‐based networks for automatic 3D multi‐organ segmentation on CT images

“…Currently, the clinical practice for contour delineation involves a labor-intensive and operator-dependent manual process. [4][5][6] The manual contouring process in addition to often being inefficient can also suffer from inconsistencies in contouring preferences or related intra-and inter-observer uncertainties. 4,7 Inaccuracies in contouring impact on planning margin design-erroneous planning margins may lead to possible underdosage of the target and excess radiation delivered to surrounding healthy tissues.…”

“…[4][5][6] The manual contouring process in addition to often being inefficient can also suffer from inconsistencies in contouring preferences or related intra-and inter-observer uncertainties. 4,7 Inaccuracies in contouring impact on planning margin design-erroneous planning margins may lead to possible underdosage of the target and excess radiation delivered to surrounding healthy tissues. 8 To address these issues, a method for accurate automatic segmentation is needed to improve efficiency and consistency in radiation treatment planning.…”

“…One of the key steps in the planning process involves segmenting the treatment target and organs‐at‐risk (OARs) typically using planning CT images. Currently, the clinical practice for contour delineation involves a labor‐intensive and operator‐dependent manual process 4–6 . The manual contouring process in addition to often being inefficient can also suffer from inconsistencies in contouring preferences or related intra‐and inter‐observer uncertainties 4,7 .…”