Quality assurance and training procedures for computer‐aided detection and diagnosis systems in clinical use^a)

Abstract

Computer-aided detection/diagnosis (CAD) is increasingly used for decision support by clinicians for detection and interpretation of diseases. However, there are no quality assurance (QA) requirements for CAD in clinical use at present. QA of CAD is important so that end users can be made aware of changes in CAD performance both due to intentional or unintentional causes. In addition, enduser training is critical to prevent improper use of CAD, which could potentially result in lower overall clinical performance. Research on QA of CAD and user training are limited to date. The purpose of this paper is to bring attention to these issues, inform the readers of the opinions of the members of the American Association of Physicists in Medicine (AAPM) CAD subcommittee, and thus stimulate further discussion in the CAD community on these topics. The recommendations in this paper are intended to be work items for AAPM task groups that will be formed to address QA and user training issues on CAD in the future. The work items may serve as a framework for the discussion and eventual design of detailed QA and training procedures for physicists and users of CAD. Some of the recommendations are considered by the subcommittee to be reasonably easy and practical and can be implemented immediately by the end users; others are considered to be "best practice" approaches, which may require significant effort, additional tools, and proper training to implement. The eventual standardization of the requirements of QA procedures for CAD will have to be determined through consensus from members of the CAD community, and user training may require support of professional societies. It is expected that high-quality CAD and proper use of CAD could allow these systems to achieve their true potential, thus benefiting both the patients and the clinicians, and may bring about more widespread clinical use of CAD for many other diseases and applications. It is hoped that the awareness of the need for appropriate CAD QA and user training will stimulate new ideas and approaches for implementing such procedures efficiently and effectively as well as funding opportunities to fulfill such critical efforts.

Computer‐aided diagnosis in the era of deep learning

“…We have seen early warning signs from the sensational news on accidents by self‐driving cars, whose drivers might have ignored the warning that they should be the hands‐on drivers, or on machine recommending incorrect or unsafe cancer treatment after the initial excitement about its helpfulness. The AAPM CAD Subcommittee (renamed as AAPM Computer Aided Image Analysis Subcommittee in 2018) has published two opinion papers on the proper training and evaluation of CAD devices, and the quality assurance and user training on CAD devices in clinical use . The discussions have not attracted much attention previously but it will be timely to revisit these issues in view of the renewed interests in deep learning‐based CAD and computer‐assisted quantitative image analysis, or AI, in medical imaging, under the leadership of organizations such as the AAPM, the American College of Radiology (ACR) and the Radiological Society of North America (RSNA).…”

Physical characterization of the quality of medical images does not adequately reflect their clinical quality

“…Finally, I do concede that, for computer‐aided diagnosis (CAD), where direct participation of human observers is not necessary, instrumental techniques can be successfully utilized. Development of CAD performance requirements, QA procedures, necessary software tools and phantoms, are topics of active interest …”

“…Development of CAD performance requirements, QA procedures, necessary software tools and phantoms, are topics of active interest. 18 Rebuttal: A. Kyle Jones, Ph.D.…”

Progress in Fully Automated Abdominal CT Interpretation

“…The automated report could improve reading efficiency, but radiologists will need to be vigilant to avoid placing too much trust in the computer. Further research on human factors, such as visual perception, and on quality assurance and promotion of a safety culture will be required to understand potential failures of the technology when used in the clinical setting [183–185]. …”