Highdicom: a Python Library for Standardized Encoding of Image Annotations and Machine Learning Model Outputs in Pathology and Radiology

Abstract

Machine learning (ML) is revolutionizing image-based diagnostics in pathology and radiology. ML models have shown promising results in research settings, but the lack of interoperability between ML systems and enterprise medical imaging systems has been a major barrier for clinical integration and evaluation. The DICOM® standard specifies information object definitions (IODs) and services for the representation and communication of digital images and related information, including image-derived annotations and analysis results. However, the complexity of the standard represents an obstacle for its adoption in the ML community and creates a need for software libraries and tools that simplify working with datasets in DICOM format. Here we present the highdicom library, which provides a high-level application programming interface (API) for the Python programming language that abstracts low-level details of the standard and enables encoding and decoding of image-derived information in DICOM format in a few lines of Python code. The highdicom library leverages NumPy arrays for efficient data representation and ties into the extensive Python ecosystem for image processing and machine learning. Simultaneously, by simplifying creation and parsing of DICOM-compliant files, highdicom achieves interoperability with the medical imaging systems that hold the data used to train and run ML models, and ultimately communicate and store model outputs for clinical use. We demonstrate through experiments with slide microscopy and computed tomography imaging, that, by bridging these two ecosystems, highdicom enables developers and researchers to train and evaluate state-of-the-art ML models in pathology and radiology while remaining compliant with the DICOM standard and interoperable with clinical systems at all stages. To promote standardization of ML research and streamline the ML model development and deployment process, we made the library available free and open-source at https://github.com/herrmannlab/highdicom.

Interoperable slide microscopy viewer and annotation tool for imaging data science and computational pathology

“…When a user ultimately saves a collection of ROI annotations, the graphic data and associated measurements and qualitative evaluations of each ROI are encoded in a DICOM Comprehensive 3D SR document using SR template “Planar ROI Measurements and Qualitative Evaluations” and the complete SR document is stored via DICOMweb. The SR template is extensible and allows for the inclusion of an unlimited number of qualitative evaluations or measurements per ROI using a multitude of coding schemes, thereby providing sufficient flexibility to support a wide range of projects and annotation use cases, while also providing a well-defined schema to facilitate the collection of structured, machine-readable annotations that can be readily used for ML model training or validation 17 , 22 , 26 . Notably, the same template is used for capturing annotations and measurements for radiology images 22 , 26 and can also be used for storing ML model outputs 26 .…”

“…The SR template is extensible and allows for the inclusion of an unlimited number of qualitative evaluations or measurements per ROI using a multitude of coding schemes, thereby providing sufficient flexibility to support a wide range of projects and annotation use cases, while also providing a well-defined schema to facilitate the collection of structured, machine-readable annotations that can be readily used for ML model training or validation 17 , 22 , 26 . Notably, the same template is used for capturing annotations and measurements for radiology images 22 , 26 and can also be used for storing ML model outputs 26 .

Fig.

…”

“…The reliance on the DICOM standard has further enabled the digital pathology and radiology communities to jointly develop and maintain common libraries and tools for the communication and visualization of biomedical imaging data in DICOM format in a device-independent and modality-agnostic manner 25 , 29 , 34 . Furthermore, since DICOM also defines information objects for derived image annotations and image analysis results, the use of DICOM has advantages beyond image display and enables a unified approach to imaging data management across modalities, projects, domains, departments, and institutions to promote interdisciplinary imaging data science and integrated diagnostics 26 .…”

“…In recent years, the standard has been further developed to support slide microscopy 24 , 25 , quantitative imaging 17 , 22 , and machine learning 26 , 27 , and is being adopted internationally for storage, management, and exchange of slide microscopy in diagnostic pathology 28 as well as in research and development 29 , 30 . While DICOM has been primarily designed for medical imaging in clinical practice and includes established mechanisms for ensuring high image quality required for diagnostic purposes, the standard also has several advantages for biomedical imaging research 17 , 25 , 26 . Notably, DICOM specifies profiles for data de-identification that facilitate the use of clinically acquired imaging data sets for research purposes 31 , 32 and defines digital objects for encoding and communicating image annotations and analysis results 17 , 22 , 26 , 27 .…”

“…While DICOM has been primarily designed for medical imaging in clinical practice and includes established mechanisms for ensuring high image quality required for diagnostic purposes, the standard also has several advantages for biomedical imaging research 17 , 25 , 26 . Notably, DICOM specifies profiles for data de-identification that facilitate the use of clinically acquired imaging data sets for research purposes 31 , 32 and defines digital objects for encoding and communicating image annotations and analysis results 17 , 22 , 26 , 27 . Furthermore, there is an abundance of open-source software libraries and tools that support DICOM 33 .…”